2023-04-27 16:53:28,412 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-04-27 16:53:28,637 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d 2023-04-27 16:53:28,653 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-04-27 16:53:28,653 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09, deleteOnExit=true 2023-04-27 16:53:28,653 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-04-27 16:53:28,654 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/test.cache.data in system properties and HBase conf 2023-04-27 16:53:28,655 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/hadoop.tmp.dir in system properties and HBase conf 2023-04-27 16:53:28,655 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/hadoop.log.dir in system properties and HBase conf 2023-04-27 16:53:28,656 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-04-27 16:53:28,656 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-04-27 16:53:28,657 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-04-27 16:53:28,762 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-04-27 16:53:29,209 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-04-27 16:53:29,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-04-27 16:53:29,215 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-04-27 16:53:29,215 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-04-27 16:53:29,216 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-27 16:53:29,216 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-04-27 16:53:29,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-04-27 16:53:29,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-27 16:53:29,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-27 16:53:29,218 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-04-27 16:53:29,218 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/nfs.dump.dir in system properties and HBase conf 2023-04-27 16:53:29,218 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/java.io.tmpdir in system properties and HBase conf 2023-04-27 16:53:29,219 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-27 16:53:29,219 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-04-27 16:53:29,219 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-04-27 16:53:29,823 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-27 16:53:29,827 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-27 16:53:30,333 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-04-27 16:53:30,487 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-04-27 16:53:30,505 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-27 16:53:30,544 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-04-27 16:53:30,576 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/java.io.tmpdir/Jetty_localhost_localdomain_36677_hdfs____8o214d/webapp 2023-04-27 16:53:30,726 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:36677 2023-04-27 16:53:30,734 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-27 16:53:30,734 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-27 16:53:31,417 WARN [Listener at localhost.localdomain/43755] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-27 16:53:31,538 WARN [Listener at localhost.localdomain/43755] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-27 16:53:31,557 WARN [Listener at localhost.localdomain/43755] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-27 16:53:31,565 INFO [Listener at localhost.localdomain/43755] log.Slf4jLog(67): jetty-6.1.26 2023-04-27 16:53:31,572 INFO [Listener at localhost.localdomain/43755] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/java.io.tmpdir/Jetty_localhost_44479_datanode____.n3psar/webapp 2023-04-27 16:53:31,664 INFO [Listener at localhost.localdomain/43755] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44479 2023-04-27 16:53:31,958 WARN [Listener at localhost.localdomain/43521] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-27 16:53:31,968 WARN [Listener at localhost.localdomain/43521] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-27 16:53:31,972 WARN [Listener at localhost.localdomain/43521] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-27 16:53:31,973 INFO [Listener at localhost.localdomain/43521] log.Slf4jLog(67): jetty-6.1.26 2023-04-27 16:53:31,978 INFO [Listener at localhost.localdomain/43521] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/java.io.tmpdir/Jetty_localhost_41287_datanode____oqj3sr/webapp 2023-04-27 16:53:32,060 INFO [Listener at localhost.localdomain/43521] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41287 2023-04-27 16:53:32,072 WARN [Listener at localhost.localdomain/39369] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-27 16:53:32,099 WARN [Listener at localhost.localdomain/39369] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-27 16:53:32,102 WARN [Listener at localhost.localdomain/39369] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-27 16:53:32,105 INFO [Listener at localhost.localdomain/39369] log.Slf4jLog(67): jetty-6.1.26 2023-04-27 16:53:32,114 INFO [Listener at localhost.localdomain/39369] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/java.io.tmpdir/Jetty_localhost_34737_datanode____1fh135/webapp 2023-04-27 16:53:32,232 INFO [Listener at localhost.localdomain/39369] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34737 2023-04-27 16:53:32,249 WARN [Listener at localhost.localdomain/45365] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-27 16:53:34,128 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x87035e1cadf17838: Processing first storage report for DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad from datanode 5799fd8f-0a4b-4c91-a1ff-1fe687f21837 2023-04-27 16:53:34,130 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x87035e1cadf17838: from storage DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad node DatanodeRegistration(127.0.0.1:45437, datanodeUuid=5799fd8f-0a4b-4c91-a1ff-1fe687f21837, infoPort=43791, infoSecurePort=0, ipcPort=45365, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,130 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf3b794f3073709b0: Processing first storage report for DS-b01ffe0b-c958-409e-8274-4f18d313868e from datanode c3e9854d-f6b7-4cf1-bd29-59caf4745df9 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf3b794f3073709b0: from storage DS-b01ffe0b-c958-409e-8274-4f18d313868e node DatanodeRegistration(127.0.0.1:39801, datanodeUuid=c3e9854d-f6b7-4cf1-bd29-59caf4745df9, infoPort=37535, infoSecurePort=0, ipcPort=39369, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xeb4939cbfd6b965e: Processing first storage report for DS-ea6bf07e-6968-4622-b68d-8155d89967da from datanode 6c9ebd47-0a00-4e93-9558-ec52eb9cbf04 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xeb4939cbfd6b965e: from storage DS-ea6bf07e-6968-4622-b68d-8155d89967da node DatanodeRegistration(127.0.0.1:40175, datanodeUuid=6c9ebd47-0a00-4e93-9558-ec52eb9cbf04, infoPort=38707, infoSecurePort=0, ipcPort=43521, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x87035e1cadf17838: Processing first storage report for DS-82984c2c-fa5c-4ea8-8870-0f79d324b5d9 from datanode 5799fd8f-0a4b-4c91-a1ff-1fe687f21837 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x87035e1cadf17838: from storage DS-82984c2c-fa5c-4ea8-8870-0f79d324b5d9 node DatanodeRegistration(127.0.0.1:45437, datanodeUuid=5799fd8f-0a4b-4c91-a1ff-1fe687f21837, infoPort=43791, infoSecurePort=0, ipcPort=45365, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf3b794f3073709b0: Processing first storage report for DS-9c436137-fa8c-4a20-a5b3-949f408add33 from datanode c3e9854d-f6b7-4cf1-bd29-59caf4745df9 2023-04-27 16:53:34,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf3b794f3073709b0: from storage DS-9c436137-fa8c-4a20-a5b3-949f408add33 node DatanodeRegistration(127.0.0.1:39801, datanodeUuid=c3e9854d-f6b7-4cf1-bd29-59caf4745df9, infoPort=37535, infoSecurePort=0, ipcPort=39369, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,132 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xeb4939cbfd6b965e: Processing first storage report for DS-5b274144-fc6e-4148-a3ed-f85719ff2cdb from datanode 6c9ebd47-0a00-4e93-9558-ec52eb9cbf04 2023-04-27 16:53:34,132 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xeb4939cbfd6b965e: from storage DS-5b274144-fc6e-4148-a3ed-f85719ff2cdb node DatanodeRegistration(127.0.0.1:40175, datanodeUuid=6c9ebd47-0a00-4e93-9558-ec52eb9cbf04, infoPort=38707, infoSecurePort=0, ipcPort=43521, storageInfo=lv=-57;cid=testClusterID;nsid=704982242;c=1682614409898), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-27 16:53:34,206 DEBUG [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d 2023-04-27 16:53:34,274 INFO [Listener at localhost.localdomain/45365] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/zookeeper_0, clientPort=58499, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-04-27 16:53:34,286 INFO [Listener at localhost.localdomain/45365] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58499 2023-04-27 16:53:34,293 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:34,295 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:34,943 INFO [Listener at localhost.localdomain/45365] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86 with version=8 2023-04-27 16:53:34,943 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/hbase-staging 2023-04-27 16:53:35,233 INFO [Listener at localhost.localdomain/45365] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-04-27 16:53:35,627 INFO [Listener at localhost.localdomain/45365] client.ConnectionUtils(127): master/jenkins-hbase12:0 server-side Connection retries=45 2023-04-27 16:53:35,652 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:35,653 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:35,653 INFO [Listener at localhost.localdomain/45365] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-27 16:53:35,653 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:35,653 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-27 16:53:35,771 INFO [Listener at localhost.localdomain/45365] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-04-27 16:53:35,830 DEBUG [Listener at localhost.localdomain/45365] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-04-27 16:53:35,909 INFO [Listener at localhost.localdomain/45365] ipc.NettyRpcServer(120): Bind to /136.243.104.168:37667 2023-04-27 16:53:35,919 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:35,920 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:35,937 INFO [Listener at localhost.localdomain/45365] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37667 connecting to ZooKeeper ensemble=127.0.0.1:58499 2023-04-27 16:53:36,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:376670x0, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-27 16:53:36,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37667-0x100f3748f680000 connected 2023-04-27 16:53:36,191 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-27 16:53:36,192 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:53:36,195 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-27 16:53:36,203 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37667 2023-04-27 16:53:36,203 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37667 2023-04-27 16:53:36,204 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37667 2023-04-27 16:53:36,205 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37667 2023-04-27 16:53:36,205 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37667 2023-04-27 16:53:36,210 INFO [Listener at localhost.localdomain/45365] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86, hbase.cluster.distributed=false 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,270 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-27 16:53:36,274 INFO [Listener at localhost.localdomain/45365] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-27 16:53:36,277 INFO [Listener at localhost.localdomain/45365] ipc.NettyRpcServer(120): Bind to /136.243.104.168:44953 2023-04-27 16:53:36,279 INFO [Listener at localhost.localdomain/45365] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-27 16:53:36,285 DEBUG [Listener at localhost.localdomain/45365] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-27 16:53:36,286 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,288 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,290 INFO [Listener at localhost.localdomain/45365] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44953 connecting to ZooKeeper ensemble=127.0.0.1:58499 2023-04-27 16:53:36,306 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:449530x0, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-27 16:53:36,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44953-0x100f3748f680001 connected 2023-04-27 16:53:36,308 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-27 16:53:36,310 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:53:36,311 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-27 16:53:36,312 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44953 2023-04-27 16:53:36,313 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44953 2023-04-27 16:53:36,316 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44953 2023-04-27 16:53:36,317 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44953 2023-04-27 16:53:36,317 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44953 2023-04-27 16:53:36,328 INFO [Listener at localhost.localdomain/45365] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-04-27 16:53:36,328 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,329 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,329 INFO [Listener at localhost.localdomain/45365] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-27 16:53:36,329 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,330 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-27 16:53:36,330 INFO [Listener at localhost.localdomain/45365] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-27 16:53:36,333 INFO [Listener at localhost.localdomain/45365] ipc.NettyRpcServer(120): Bind to /136.243.104.168:41143 2023-04-27 16:53:36,333 INFO [Listener at localhost.localdomain/45365] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-27 16:53:36,336 DEBUG [Listener at localhost.localdomain/45365] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-27 16:53:36,337 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,339 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,343 INFO [Listener at localhost.localdomain/45365] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41143 connecting to ZooKeeper ensemble=127.0.0.1:58499 2023-04-27 16:53:36,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:411430x0, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-27 16:53:36,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41143-0x100f3748f680002 connected 2023-04-27 16:53:36,360 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-27 16:53:36,361 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:53:36,362 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-27 16:53:36,363 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41143 2023-04-27 16:53:36,363 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41143 2023-04-27 16:53:36,364 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41143 2023-04-27 16:53:36,364 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41143 2023-04-27 16:53:36,365 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41143 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-27 16:53:36,375 INFO [Listener at localhost.localdomain/45365] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-27 16:53:36,377 INFO [Listener at localhost.localdomain/45365] ipc.NettyRpcServer(120): Bind to /136.243.104.168:46011 2023-04-27 16:53:36,377 INFO [Listener at localhost.localdomain/45365] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-27 16:53:36,378 DEBUG [Listener at localhost.localdomain/45365] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-27 16:53:36,379 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,381 INFO [Listener at localhost.localdomain/45365] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,382 INFO [Listener at localhost.localdomain/45365] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46011 connecting to ZooKeeper ensemble=127.0.0.1:58499 2023-04-27 16:53:36,390 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:460110x0, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-27 16:53:36,391 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:460110x0, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-27 16:53:36,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46011-0x100f3748f680003 connected 2023-04-27 16:53:36,392 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:53:36,393 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-27 16:53:36,394 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46011 2023-04-27 16:53:36,394 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46011 2023-04-27 16:53:36,394 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46011 2023-04-27 16:53:36,395 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46011 2023-04-27 16:53:36,395 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46011 2023-04-27 16:53:36,397 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:36,412 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-27 16:53:36,413 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:36,432 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-27 16:53:36,432 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-27 16:53:36,432 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-27 16:53:36,432 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-27 16:53:36,432 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:36,434 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-27 16:53:36,435 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase12.apache.org,37667,1682614415067 from backup master directory 2023-04-27 16:53:36,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-27 16:53:36,442 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:36,443 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-27 16:53:36,443 WARN [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-27 16:53:36,444 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:36,447 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-04-27 16:53:36,448 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-04-27 16:53:36,547 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/hbase.id with ID: f5231fca-0252-4387-879f-1a8ad1de9151 2023-04-27 16:53:36,590 INFO [master/jenkins-hbase12:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-27 16:53:36,611 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:36,658 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x73ce194e to 127.0.0.1:58499 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-27 16:53:36,699 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d20366e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-27 16:53:36,721 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-27 16:53:36,723 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-04-27 16:53:36,738 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-04-27 16:53:36,739 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-04-27 16:53:36,741 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:53:36,745 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:53:36,746 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-27 16:53:36,778 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store-tmp 2023-04-27 16:53:36,835 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:36,835 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-27 16:53:36,835 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:53:36,835 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:53:36,835 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-27 16:53:36,835 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:53:36,836 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:53:36,836 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-27 16:53:36,838 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/WALs/jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:36,859 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C37667%2C1682614415067, suffix=, logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/WALs/jenkins-hbase12.apache.org,37667,1682614415067, archiveDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/oldWALs, maxLogs=10 2023-04-27 16:53:36,911 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK] 2023-04-27 16:53:36,911 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK] 2023-04-27 16:53:36,911 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK] 2023-04-27 16:53:36,919 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:53:36,986 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/WALs/jenkins-hbase12.apache.org,37667,1682614415067/jenkins-hbase12.apache.org%2C37667%2C1682614415067.1682614416868 2023-04-27 16:53:36,988 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK], DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK], DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK]] 2023-04-27 16:53:36,989 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:36,989 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:36,993 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:36,994 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:37,056 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:37,063 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-04-27 16:53:37,086 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-04-27 16:53:37,102 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:37,109 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:37,111 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:37,126 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-04-27 16:53:37,131 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:37,133 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72142901, jitterRate=0.07501299679279327}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-27 16:53:37,133 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-27 16:53:37,134 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-04-27 16:53:37,151 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:37,151 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:37,153 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:37,155 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-27 16:53:37,182 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 26 msec 2023-04-27 16:53:37,182 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-04-27 16:53:37,203 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-04-27 16:53:37,208 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-04-27 16:53:37,228 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-04-27 16:53:37,232 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-04-27 16:53:37,233 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-04-27 16:53:37,237 INFO [master/jenkins-hbase12:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-04-27 16:53:37,241 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-04-27 16:53:37,295 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:37,297 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-04-27 16:53:37,297 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-04-27 16:53:37,309 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-04-27 16:53:37,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-27 16:53:37,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-27 16:53:37,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-27 16:53:37,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-27 16:53:37,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:37,317 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase12.apache.org,37667,1682614415067, sessionid=0x100f3748f680000, setting cluster-up flag (Was=false) 2023-04-27 16:53:37,347 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:37,379 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-04-27 16:53:37,381 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:37,400 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:37,432 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-04-27 16:53:37,434 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:37,436 WARN [master/jenkins-hbase12:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.hbase-snapshot/.tmp 2023-04-27 16:53:37,500 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(952): ClusterId : f5231fca-0252-4387-879f-1a8ad1de9151 2023-04-27 16:53:37,500 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(952): ClusterId : f5231fca-0252-4387-879f-1a8ad1de9151 2023-04-27 16:53:37,500 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(952): ClusterId : f5231fca-0252-4387-879f-1a8ad1de9151 2023-04-27 16:53:37,505 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-27 16:53:37,505 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-27 16:53:37,505 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-27 16:53:37,528 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-27 16:53:37,528 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-27 16:53:37,528 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-27 16:53:37,528 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-27 16:53:37,528 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-27 16:53:37,528 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-27 16:53:37,544 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:37,548 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-27 16:53:37,548 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-27 16:53:37,548 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-27 16:53:37,551 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ReadOnlyZKClient(139): Connect 0x4aa81090 to 127.0.0.1:58499 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-27 16:53:37,551 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ReadOnlyZKClient(139): Connect 0x71054da7 to 127.0.0.1:58499 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-27 16:53:37,551 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ReadOnlyZKClient(139): Connect 0x0983c9a4 to 127.0.0.1:58499 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase12:0, corePoolSize=10, maxPoolSize=10 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-04-27 16:53:37,560 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,563 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614447563 2023-04-27 16:53:37,565 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-04-27 16:53:37,570 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:37,571 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-04-27 16:53:37,576 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-27 16:53:37,576 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-04-27 16:53:37,581 DEBUG [RS:2;jenkins-hbase12:46011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fc2bdf7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-27 16:53:37,581 DEBUG [RS:1;jenkins-hbase12:41143] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@446fe7b5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-27 16:53:37,582 DEBUG [RS:2;jenkins-hbase12:46011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bfd26d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-04-27 16:53:37,582 DEBUG [RS:0;jenkins-hbase12:44953] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74436e4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-27 16:53:37,582 DEBUG [RS:1;jenkins-hbase12:41143] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40507be0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-04-27 16:53:37,583 DEBUG [RS:0;jenkins-hbase12:44953] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37ba2d4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-04-27 16:53:37,586 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-04-27 16:53:37,586 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-04-27 16:53:37,586 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-04-27 16:53:37,586 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-04-27 16:53:37,587 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,589 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-04-27 16:53:37,590 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-04-27 16:53:37,590 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-04-27 16:53:37,593 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-04-27 16:53:37,593 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-04-27 16:53:37,596 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1682614417595,5,FailOnTimeoutGroup] 2023-04-27 16:53:37,596 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1682614417596,5,FailOnTimeoutGroup] 2023-04-27 16:53:37,596 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,596 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-04-27 16:53:37,598 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,598 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,604 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase12:41143 2023-04-27 16:53:37,605 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase12:44953 2023-04-27 16:53:37,606 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase12:46011 2023-04-27 16:53:37,609 INFO [RS:1;jenkins-hbase12:41143] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-27 16:53:37,609 INFO [RS:0;jenkins-hbase12:44953] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-27 16:53:37,610 INFO [RS:0;jenkins-hbase12:44953] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-27 16:53:37,610 INFO [RS:2;jenkins-hbase12:46011] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-27 16:53:37,610 INFO [RS:2;jenkins-hbase12:46011] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-27 16:53:37,610 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1023): About to register with Master. 2023-04-27 16:53:37,610 INFO [RS:1;jenkins-hbase12:41143] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-27 16:53:37,610 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1023): About to register with Master. 2023-04-27 16:53:37,610 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1023): About to register with Master. 2023-04-27 16:53:37,614 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37667,1682614415067 with isa=jenkins-hbase12.apache.org/136.243.104.168:44953, startcode=1682614416269 2023-04-27 16:53:37,614 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37667,1682614415067 with isa=jenkins-hbase12.apache.org/136.243.104.168:41143, startcode=1682614416328 2023-04-27 16:53:37,614 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37667,1682614415067 with isa=jenkins-hbase12.apache.org/136.243.104.168:46011, startcode=1682614416374 2023-04-27 16:53:37,631 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:37,632 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:37,633 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86 2023-04-27 16:53:37,643 DEBUG [RS:1;jenkins-hbase12:41143] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-27 16:53:37,643 DEBUG [RS:0;jenkins-hbase12:44953] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-27 16:53:37,643 DEBUG [RS:2;jenkins-hbase12:46011] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-27 16:53:37,661 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:37,665 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-27 16:53:37,668 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/info 2023-04-27 16:53:37,669 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-27 16:53:37,670 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:37,671 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-27 16:53:37,675 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/rep_barrier 2023-04-27 16:53:37,676 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-27 16:53:37,677 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:37,678 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-27 16:53:37,680 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/table 2023-04-27 16:53:37,681 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-27 16:53:37,682 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:37,687 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740 2023-04-27 16:53:37,688 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740 2023-04-27 16:53:37,695 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-27 16:53:37,697 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-27 16:53:37,702 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:37,704 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63020852, jitterRate=-0.060916125774383545}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-27 16:53:37,704 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-27 16:53:37,704 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-27 16:53:37,704 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-27 16:53:37,704 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-27 16:53:37,704 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-27 16:53:37,704 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-27 16:53:37,711 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:60557, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-04-27 16:53:37,711 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:42081, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-04-27 16:53:37,711 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-27 16:53:37,711 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:40873, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-04-27 16:53:37,712 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-27 16:53:37,718 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:37,718 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-04-27 16:53:37,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37667] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-04-27 16:53:37,744 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86 2023-04-27 16:53:37,744 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86 2023-04-27 16:53:37,744 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86 2023-04-27 16:53:37,744 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43755 2023-04-27 16:53:37,744 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-27 16:53:37,744 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43755 2023-04-27 16:53:37,744 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-27 16:53:37,744 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43755 2023-04-27 16:53:37,744 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-27 16:53:37,753 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-04-27 16:53:37,755 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-27 16:53:37,758 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:53:37,800 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ZKUtil(162): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,801 WARN [RS:2;jenkins-hbase12:46011] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-27 16:53:37,801 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ZKUtil(162): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,801 INFO [RS:2;jenkins-hbase12:46011] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-27 16:53:37,801 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ZKUtil(162): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,801 WARN [RS:1;jenkins-hbase12:41143] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-27 16:53:37,802 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,802 INFO [RS:1;jenkins-hbase12:41143] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-27 16:53:37,802 WARN [RS:0;jenkins-hbase12:44953] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-27 16:53:37,802 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,802 INFO [RS:0;jenkins-hbase12:44953] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-27 16:53:37,802 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,44953,1682614416269] 2023-04-27 16:53:37,802 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,46011,1682614416374] 2023-04-27 16:53:37,802 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,41143,1682614416328] 2023-04-27 16:53:37,802 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,813 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ZKUtil(162): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,813 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ZKUtil(162): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,813 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ZKUtil(162): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,814 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ZKUtil(162): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,814 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ZKUtil(162): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,814 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ZKUtil(162): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,814 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ZKUtil(162): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,814 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ZKUtil(162): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,815 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ZKUtil(162): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,823 DEBUG [RS:0;jenkins-hbase12:44953] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-27 16:53:37,823 DEBUG [RS:1;jenkins-hbase12:41143] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-27 16:53:37,823 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-27 16:53:37,832 INFO [RS:2;jenkins-hbase12:46011] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-27 16:53:37,833 INFO [RS:1;jenkins-hbase12:41143] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-27 16:53:37,832 INFO [RS:0;jenkins-hbase12:44953] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-27 16:53:37,852 INFO [RS:1;jenkins-hbase12:41143] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-27 16:53:37,852 INFO [RS:2;jenkins-hbase12:46011] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-27 16:53:37,852 INFO [RS:0;jenkins-hbase12:44953] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-27 16:53:37,857 INFO [RS:1;jenkins-hbase12:41143] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-27 16:53:37,857 INFO [RS:0;jenkins-hbase12:44953] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-27 16:53:37,857 INFO [RS:2;jenkins-hbase12:46011] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-27 16:53:37,857 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,858 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,857 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,859 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-27 16:53:37,859 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-27 16:53:37,859 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-27 16:53:37,867 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,867 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,867 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,867 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,867 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,868 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-04-27 16:53:37,868 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:2;jenkins-hbase12:46011] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:0;jenkins-hbase12:44953] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,869 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,870 DEBUG [RS:1;jenkins-hbase12:41143] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-04-27 16:53:37,872 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,872 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,872 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,873 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,873 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,873 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,877 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,877 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,877 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,888 INFO [RS:0;jenkins-hbase12:44953] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-27 16:53:37,891 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,44953,1682614416269-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,892 INFO [RS:2;jenkins-hbase12:46011] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-27 16:53:37,893 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,46011,1682614416374-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,903 INFO [RS:1;jenkins-hbase12:41143] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-27 16:53:37,904 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,41143,1682614416328-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:37,908 DEBUG [jenkins-hbase12:37667] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-04-27 16:53:37,914 INFO [RS:0;jenkins-hbase12:44953] regionserver.Replication(203): jenkins-hbase12.apache.org,44953,1682614416269 started 2023-04-27 16:53:37,914 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,44953,1682614416269, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:44953, sessionid=0x100f3748f680001 2023-04-27 16:53:37,917 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-27 16:53:37,917 DEBUG [RS:0;jenkins-hbase12:44953] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,917 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,44953,1682614416269' 2023-04-27 16:53:37,917 INFO [RS:1;jenkins-hbase12:41143] regionserver.Replication(203): jenkins-hbase12.apache.org,41143,1682614416328 started 2023-04-27 16:53:37,917 INFO [RS:2;jenkins-hbase12:46011] regionserver.Replication(203): jenkins-hbase12.apache.org,46011,1682614416374 started 2023-04-27 16:53:37,917 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,41143,1682614416328, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:41143, sessionid=0x100f3748f680002 2023-04-27 16:53:37,917 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:37,917 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-27 16:53:37,917 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,46011,1682614416374, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:46011, sessionid=0x100f3748f680003 2023-04-27 16:53:37,917 DEBUG [RS:1;jenkins-hbase12:41143] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,918 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-27 16:53:37,919 DEBUG [RS:2;jenkins-hbase12:46011] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,919 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,46011,1682614416374' 2023-04-27 16:53:37,919 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:37,918 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,41143,1682614416328' 2023-04-27 16:53:37,919 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:37,918 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:53:37,920 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:37,920 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:37,920 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-27 16:53:37,920 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-27 16:53:37,920 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-27 16:53:37,920 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-27 16:53:37,921 DEBUG [RS:1;jenkins-hbase12:41143] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:37,921 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:37,921 DEBUG [RS:2;jenkins-hbase12:46011] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:37,922 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,46011,1682614416374' 2023-04-27 16:53:37,922 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-27 16:53:37,921 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,41143,1682614416328' 2023-04-27 16:53:37,922 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-27 16:53:37,922 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-27 16:53:37,922 DEBUG [RS:1;jenkins-hbase12:41143] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-27 16:53:37,922 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-27 16:53:37,922 DEBUG [RS:0;jenkins-hbase12:44953] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:37,923 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,44953,1682614416269' 2023-04-27 16:53:37,923 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-27 16:53:37,923 DEBUG [RS:1;jenkins-hbase12:41143] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-27 16:53:37,923 INFO [RS:1;jenkins-hbase12:41143] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-27 16:53:37,923 INFO [RS:1;jenkins-hbase12:41143] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-27 16:53:37,923 DEBUG [RS:0;jenkins-hbase12:44953] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-27 16:53:37,924 DEBUG [RS:2;jenkins-hbase12:46011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-27 16:53:37,924 DEBUG [RS:0;jenkins-hbase12:44953] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-27 16:53:37,924 INFO [RS:0;jenkins-hbase12:44953] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-27 16:53:37,925 INFO [RS:0;jenkins-hbase12:44953] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-27 16:53:37,927 DEBUG [RS:2;jenkins-hbase12:46011] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-27 16:53:37,927 INFO [RS:2;jenkins-hbase12:46011] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-27 16:53:37,927 INFO [RS:2;jenkins-hbase12:46011] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-27 16:53:37,930 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:53:37,930 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:53:37,930 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:53:37,930 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:53:37,934 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,46011,1682614416374, state=OPENING 2023-04-27 16:53:38,015 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-04-27 16:53:38,033 INFO [RS:0;jenkins-hbase12:44953] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C44953%2C1682614416269, suffix=, logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,44953,1682614416269, archiveDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs, maxLogs=32 2023-04-27 16:53:38,033 INFO [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C46011%2C1682614416374, suffix=, logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374, archiveDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs, maxLogs=32 2023-04-27 16:53:38,033 INFO [RS:1;jenkins-hbase12:41143] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C41143%2C1682614416328, suffix=, logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,41143,1682614416328, archiveDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs, maxLogs=32 2023-04-27 16:53:38,042 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:38,043 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-27 16:53:38,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:38,069 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK] 2023-04-27 16:53:38,070 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK] 2023-04-27 16:53:38,072 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK] 2023-04-27 16:53:38,072 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK] 2023-04-27 16:53:38,073 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK] 2023-04-27 16:53:38,073 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK] 2023-04-27 16:53:38,073 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK] 2023-04-27 16:53:38,074 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK] 2023-04-27 16:53:38,074 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK] 2023-04-27 16:53:38,088 INFO [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374/jenkins-hbase12.apache.org%2C46011%2C1682614416374.1682614418038 2023-04-27 16:53:38,088 INFO [RS:1;jenkins-hbase12:41143] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,41143,1682614416328/jenkins-hbase12.apache.org%2C41143%2C1682614416328.1682614418038 2023-04-27 16:53:38,088 INFO [RS:0;jenkins-hbase12:44953] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,44953,1682614416269/jenkins-hbase12.apache.org%2C44953%2C1682614416269.1682614418038 2023-04-27 16:53:38,092 DEBUG [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK], DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK], DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK]] 2023-04-27 16:53:38,092 DEBUG [RS:1;jenkins-hbase12:41143] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK], DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK], DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK]] 2023-04-27 16:53:38,092 DEBUG [RS:0;jenkins-hbase12:44953] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK], DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK], DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK]] 2023-04-27 16:53:38,257 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:38,259 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-27 16:53:38,262 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:44532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-27 16:53:38,275 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-04-27 16:53:38,276 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-27 16:53:38,281 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C46011%2C1682614416374.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374, archiveDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs, maxLogs=32 2023-04-27 16:53:38,301 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK] 2023-04-27 16:53:38,301 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK] 2023-04-27 16:53:38,303 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK] 2023-04-27 16:53:38,308 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374/jenkins-hbase12.apache.org%2C46011%2C1682614416374.meta.1682614418282.meta 2023-04-27 16:53:38,309 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39801,DS-b01ffe0b-c958-409e-8274-4f18d313868e,DISK], DatanodeInfoWithStorage[127.0.0.1:45437,DS-9ecc2c53-5a8f-4a68-84e5-4e19dd862bad,DISK], DatanodeInfoWithStorage[127.0.0.1:40175,DS-ea6bf07e-6968-4622-b68d-8155d89967da,DISK]] 2023-04-27 16:53:38,309 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:38,313 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:38,314 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-04-27 16:53:38,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-04-27 16:53:38,334 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-04-27 16:53:38,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-04-27 16:53:38,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:38,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-04-27 16:53:38,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-04-27 16:53:38,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-27 16:53:38,343 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/info 2023-04-27 16:53:38,343 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/info 2023-04-27 16:53:38,344 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-27 16:53:38,345 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:38,345 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-27 16:53:38,346 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/rep_barrier 2023-04-27 16:53:38,346 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/rep_barrier 2023-04-27 16:53:38,347 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-27 16:53:38,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:38,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-27 16:53:38,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/table 2023-04-27 16:53:38,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/table 2023-04-27 16:53:38,350 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-27 16:53:38,351 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:38,353 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740 2023-04-27 16:53:38,356 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740 2023-04-27 16:53:38,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-27 16:53:38,362 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-27 16:53:38,364 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65675253, jitterRate=-0.021362468600273132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-27 16:53:38,364 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-27 16:53:38,378 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1682614418250 2023-04-27 16:53:38,398 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-04-27 16:53:38,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-04-27 16:53:38,400 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,46011,1682614416374, state=OPEN 2023-04-27 16:53:38,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-04-27 16:53:38,485 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-27 16:53:38,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-04-27 16:53:38,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,46011,1682614416374 in 439 msec 2023-04-27 16:53:38,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-04-27 16:53:38,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 762 msec 2023-04-27 16:53:38,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.0230 sec 2023-04-27 16:53:38,502 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1682614418501, completionTime=-1 2023-04-27 16:53:38,502 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-04-27 16:53:38,502 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:38,569 DEBUG [hconnection-0x2ad4950d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-27 16:53:38,573 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:44544, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-27 16:53:38,589 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:38,589 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614478589 2023-04-27 16:53:38,589 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614538589 2023-04-27 16:53:38,589 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 87 msec 2023-04-27 16:53:38,709 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37667,1682614415067-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:38,709 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37667,1682614415067-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:38,709 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37667,1682614415067-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:38,711 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase12:37667, period=300000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:38,711 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-04-27 16:53:38,718 DEBUG [master/jenkins-hbase12:0.Chore.1] janitor.CatalogJanitor(175): 2023-04-27 16:53:38,726 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-04-27 16:53:38,727 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-27 16:53:38,738 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:38,741 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:53:38,744 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:53:38,769 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:38,772 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf empty. 2023-04-27 16:53:38,772 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:38,773 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-04-27 16:53:38,819 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:38,821 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => a181d38dc7ff4539b60029e08bf407bf, NAME => 'hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:53:38,845 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:38,845 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing a181d38dc7ff4539b60029e08bf407bf, disabling compactions & flushes 2023-04-27 16:53:38,845 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:38,845 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:38,845 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. after waiting 0 ms 2023-04-27 16:53:38,845 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:38,846 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:38,846 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for a181d38dc7ff4539b60029e08bf407bf: 2023-04-27 16:53:38,851 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:53:38,866 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682614418855"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614418855"}]},"ts":"1682614418855"} 2023-04-27 16:53:38,892 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:53:38,894 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:53:38,899 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614418894"}]},"ts":"1682614418894"} 2023-04-27 16:53:38,904 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-04-27 16:53:38,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:53:38,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:53:38,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:53:38,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:53:38,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:53:38,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a181d38dc7ff4539b60029e08bf407bf, ASSIGN}] 2023-04-27 16:53:38,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a181d38dc7ff4539b60029e08bf407bf, ASSIGN 2023-04-27 16:53:38,938 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a181d38dc7ff4539b60029e08bf407bf, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:39,090 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:53:39,091 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a181d38dc7ff4539b60029e08bf407bf, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:39,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682614419091"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614419091"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614419091"}]},"ts":"1682614419091"} 2023-04-27 16:53:39,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a181d38dc7ff4539b60029e08bf407bf, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:39,256 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:39,257 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a181d38dc7ff4539b60029e08bf407bf, NAME => 'hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:39,258 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:39,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:39,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,259 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,262 INFO [StoreOpener-a181d38dc7ff4539b60029e08bf407bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,264 DEBUG [StoreOpener-a181d38dc7ff4539b60029e08bf407bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/info 2023-04-27 16:53:39,264 DEBUG [StoreOpener-a181d38dc7ff4539b60029e08bf407bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/info 2023-04-27 16:53:39,265 INFO [StoreOpener-a181d38dc7ff4539b60029e08bf407bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a181d38dc7ff4539b60029e08bf407bf columnFamilyName info 2023-04-27 16:53:39,266 INFO [StoreOpener-a181d38dc7ff4539b60029e08bf407bf-1] regionserver.HStore(310): Store=a181d38dc7ff4539b60029e08bf407bf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:39,268 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,270 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,274 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:53:39,277 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:39,278 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened a181d38dc7ff4539b60029e08bf407bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61312231, jitterRate=-0.08637656271457672}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-27 16:53:39,278 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for a181d38dc7ff4539b60029e08bf407bf: 2023-04-27 16:53:39,281 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf., pid=6, masterSystemTime=1682614419249 2023-04-27 16:53:39,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:39,284 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:53:39,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:39,286 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1682614419285"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614419285"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614419285"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614419285"}]},"ts":"1682614419285"} 2023-04-27 16:53:39,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-04-27 16:53:39,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a181d38dc7ff4539b60029e08bf407bf, server=jenkins-hbase12.apache.org,46011,1682614416374 in 195 msec 2023-04-27 16:53:39,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-04-27 16:53:39,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a181d38dc7ff4539b60029e08bf407bf, ASSIGN in 362 msec 2023-04-27 16:53:39,299 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:53:39,299 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614419299"}]},"ts":"1682614419299"} 2023-04-27 16:53:39,301 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-04-27 16:53:39,349 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-04-27 16:53:39,350 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:53:39,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 622 msec 2023-04-27 16:53:39,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-04-27 16:53:39,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:39,390 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:39,411 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-27 16:53:39,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 43 msec 2023-04-27 16:53:39,434 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:39,453 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-27 16:53:39,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 33 msec 2023-04-27 16:53:39,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-04-27 16:53:39,516 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-04-27 16:53:39,518 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.074sec 2023-04-27 16:53:39,520 INFO [master/jenkins-hbase12:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-04-27 16:53:39,521 INFO [master/jenkins-hbase12:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-04-27 16:53:39,522 INFO [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-04-27 16:53:39,523 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37667,1682614415067-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-04-27 16:53:39,524 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37667,1682614415067-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-04-27 16:53:39,532 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-04-27 16:53:39,606 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ReadOnlyZKClient(139): Connect 0x0ea999b8 to 127.0.0.1:58499 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-27 16:53:39,622 DEBUG [Listener at localhost.localdomain/45365] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@394dd04b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-27 16:53:39,636 DEBUG [hconnection-0x6d42571f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-27 16:53:39,648 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:44546, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-27 16:53:39,658 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:53:39,685 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=359, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=403, ProcessCount=174, AvailableMemoryMB=12022 2023-04-27 16:53:39,689 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:39,698 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-27 16:53:39,702 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:45616, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-27 16:53:39,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:53:39,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:53:39,728 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:53:39,734 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:39,737 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:53:39,739 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:53:39,742 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:39,742 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b empty. 2023-04-27 16:53:39,745 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:39,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-04-27 16:53:39,770 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:39,796 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ce00783d83d84715d1e04b95abdb46b, NAME => 'testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:53:39,822 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:39,822 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 6ce00783d83d84715d1e04b95abdb46b, disabling compactions & flushes 2023-04-27 16:53:39,823 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:39,823 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:39,823 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. after waiting 0 ms 2023-04-27 16:53:39,823 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:39,823 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:39,823 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:39,828 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:53:39,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614419829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614419829"}]},"ts":"1682614419829"} 2023-04-27 16:53:39,833 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:53:39,835 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:53:39,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614419835"}]},"ts":"1682614419835"} 2023-04-27 16:53:39,837 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-04-27 16:53:39,854 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:53:39,855 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:53:39,856 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:53:39,856 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:53:39,856 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:53:39,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, ASSIGN}] 2023-04-27 16:53:39,859 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, ASSIGN 2023-04-27 16:53:39,861 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:40,011 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:53:40,012 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6ce00783d83d84715d1e04b95abdb46b, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614420012"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614420012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614420012"}]},"ts":"1682614420012"} 2023-04-27 16:53:40,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 6ce00783d83d84715d1e04b95abdb46b, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:40,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:40,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ce00783d83d84715d1e04b95abdb46b, NAME => 'testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:40,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:40,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:40,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,180 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,183 DEBUG [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1 2023-04-27 16:53:40,183 DEBUG [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1 2023-04-27 16:53:40,183 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ce00783d83d84715d1e04b95abdb46b columnFamilyName cf1 2023-04-27 16:53:40,184 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] regionserver.HStore(310): Store=6ce00783d83d84715d1e04b95abdb46b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:40,184 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,186 DEBUG [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2 2023-04-27 16:53:40,186 DEBUG [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2 2023-04-27 16:53:40,187 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ce00783d83d84715d1e04b95abdb46b columnFamilyName cf2 2023-04-27 16:53:40,188 INFO [StoreOpener-6ce00783d83d84715d1e04b95abdb46b-1] regionserver.HStore(310): Store=6ce00783d83d84715d1e04b95abdb46b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:40,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:40,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:40,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:40,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 6ce00783d83d84715d1e04b95abdb46b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65033987, jitterRate=-0.030918076634407043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:40,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:40,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b., pid=11, masterSystemTime=1682614420170 2023-04-27 16:53:40,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:40,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:40,204 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6ce00783d83d84715d1e04b95abdb46b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,204 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614420204"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614420204"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614420204"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614420204"}]},"ts":"1682614420204"} 2023-04-27 16:53:40,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-04-27 16:53:40,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 6ce00783d83d84715d1e04b95abdb46b, server=jenkins-hbase12.apache.org,46011,1682614416374 in 192 msec 2023-04-27 16:53:40,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-04-27 16:53:40,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, ASSIGN in 356 msec 2023-04-27 16:53:40,217 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:53:40,217 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614420217"}]},"ts":"1682614420217"} 2023-04-27 16:53:40,220 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-04-27 16:53:40,234 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:53:40,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 502 msec 2023-04-27 16:53:40,510 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:40,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,519 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:40,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:40,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:40,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:40,520 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,520 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,568 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,570 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:40,573 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:40,574 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:40,574 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:40,574 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:40,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:40,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:40,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:40,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:40,576 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:40,576 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:40,577 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:40,577 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:40,578 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:53:40,589 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,600 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,600 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,600 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:40,600 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:40,650 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/993dc1d20d854fe2a49161b692859199 2023-04-27 16:53:40,725 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c 2023-04-27 16:53:40,735 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/993dc1d20d854fe2a49161b692859199 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199 2023-04-27 16:53:40,744 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:40,747 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c 2023-04-27 16:53:40,759 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:40,761 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 183ms, sequenceid=6, compaction requested=false 2023-04-27 16:53:40,762 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:40,765 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:40,765 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:40,765 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:40,766 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:40,766 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,766 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:40,766 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:40,779 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,779 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:40,779 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,780 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:40,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:40,782 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:40,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75c211d9[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:40,782 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:40,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:40,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:40,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:40,818 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:40,833 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:40,833 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:40,834 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:40,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:40,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:40,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:40,842 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,842 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:40,843 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:40,843 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:40,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:40,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:40,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:40,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:40,849 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:40,849 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:40,849 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:40,849 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,849 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:40,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:40,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:40,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,863 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,864 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,864 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:40,866 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,867 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,867 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:40,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:40,885 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:40,885 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:40,885 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:40,886 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:40,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:40,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:40,888 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:40,888 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:40,989 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:40,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:41,022 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:41,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:41,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,025 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,025 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,026 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:41,026 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:41,027 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,027 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,031 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,033 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,034 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,034 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:41,035 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:41,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,035 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,035 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,035 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:41,035 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,035 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,035 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,036 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:41,036 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,052 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,052 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,053 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,053 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,053 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,063 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/ea1e1d20ae284a0c9a98b31332d18b0d 2023-04-27 16:53:41,091 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/1fa73873fd294cffaa072d4fd8dd104d 2023-04-27 16:53:41,100 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/ea1e1d20ae284a0c9a98b31332d18b0d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d 2023-04-27 16:53:41,108 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:41,109 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/1fa73873fd294cffaa072d4fd8dd104d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d 2023-04-27 16:53:41,117 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:41,119 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 84ms, sequenceid=19, compaction requested=false 2023-04-27 16:53:41,119 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:41,119 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,119 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:41,120 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:41,120 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,120 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,120 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,126 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,126 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:41,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@335a68a0[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:41,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:41,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,137 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:41,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:41,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:41,138 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:41,148 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,148 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,148 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,158 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,158 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,158 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,168 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,168 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,169 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:41,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:41,174 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:41,174 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:41,174 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:41,174 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,174 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,179 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,180 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,181 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,181 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,183 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,183 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,183 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,256 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,256 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,257 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,255 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,256 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,257 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,257 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:41,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,258 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,358 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:41,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:41,389 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:41,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:41,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,392 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,392 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,392 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:41,392 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:41,393 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,393 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,410 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,411 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,410 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,410 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,413 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,413 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,413 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,413 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,413 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,415 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,413 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,415 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,415 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,416 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,416 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,416 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,415 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,416 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,416 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,417 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,417 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,417 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,417 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,417 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:41,418 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:41,418 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,418 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:41,418 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:41,431 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,435 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 2023-04-27 16:53:41,442 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,442 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,442 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,442 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,442 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,464 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/973140fb6efb41738e96cd3f398a06a0 2023-04-27 16:53:41,473 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 2023-04-27 16:53:41,482 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:41,484 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/973140fb6efb41738e96cd3f398a06a0 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0 2023-04-27 16:53:41,493 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:41,494 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 76ms, sequenceid=32, compaction requested=true 2023-04-27 16:53:41,494 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:41,494 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,495 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:41,495 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:41,495 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,495 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,495 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,505 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,505 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,506 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,506 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:41,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2661499d[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:41,508 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:41,508 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,516 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,516 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,516 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,517 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,516 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,517 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,517 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,517 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:41,517 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:41,528 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,528 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,528 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,537 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,537 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,537 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,537 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,537 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,537 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,538 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:41,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:41,544 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:41,544 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:41,544 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:41,544 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,544 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,558 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,559 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,559 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,559 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,562 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,562 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,563 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,569 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,569 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,569 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,569 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,569 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:41,569 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,569 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,569 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,570 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,670 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:41,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:41,700 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:41,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:41,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,703 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,703 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,704 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:41,704 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:41,705 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,705 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,718 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:41,718 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,719 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,719 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,719 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,719 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,720 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,719 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:41,720 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:41,720 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,720 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:41,721 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:41,721 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:41,721 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,721 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,721 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:41,721 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,721 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:41,726 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,738 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,739 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,738 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,740 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,744 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/6fbe84df577a449ebb8c5cb11eb79679 2023-04-27 16:53:41,772 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/69a08e6a329d488ba0aa6c07c329d4c0 2023-04-27 16:53:41,780 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/6fbe84df577a449ebb8c5cb11eb79679 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679 2023-04-27 16:53:41,788 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:41,790 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/69a08e6a329d488ba0aa6c07c329d4c0 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0 2023-04-27 16:53:41,798 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:41,800 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 79ms, sequenceid=45, compaction requested=true 2023-04-27 16:53:41,800 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:41,800 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:41,800 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:41,802 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:41,802 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,802 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:41,802 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:41,840 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,840 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,841 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,841 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:41,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:41,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fe9e09c[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:41,843 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:41,843 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:41,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,874 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,874 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,875 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,876 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:41,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:41,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:41,879 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:41,879 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:41,879 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:41,879 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,879 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,884 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,885 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:41,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,885 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:41,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,886 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,888 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:41,888 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,888 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,895 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:41,895 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:41,895 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:41,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:41,895 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,896 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,896 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,896 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:41,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:41,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,897 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:41,997 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:41,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:42,024 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:42,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:42,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,027 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:42,027 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,028 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:42,028 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:42,028 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,028 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,031 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,032 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,035 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,035 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,036 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,037 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:42,037 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:42,037 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,037 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:42,037 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:42,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,042 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,043 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,043 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,043 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,043 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,054 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/570ffb34590f4add800e3e49db77a93c 2023-04-27 16:53:42,081 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/0d732736c3814019816c6a04a8e9f962 2023-04-27 16:53:42,089 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/570ffb34590f4add800e3e49db77a93c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c 2023-04-27 16:53:42,098 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:42,099 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/0d732736c3814019816c6a04a8e9f962 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962 2023-04-27 16:53:42,107 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:42,108 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 71ms, sequenceid=58, compaction requested=true 2023-04-27 16:53:42,108 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:42,108 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,108 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:42,109 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:42,109 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,109 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,109 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,116 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:42,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ac07e61[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:42,118 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:42,118 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,127 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,127 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:42,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:42,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,147 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,147 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,147 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,148 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,158 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,158 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,158 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:42,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:42,162 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:42,163 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:42,163 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:42,163 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,163 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:42,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,169 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,170 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,171 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,171 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,171 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,209 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:42,210 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,210 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,209 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,210 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:42,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,210 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,211 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,212 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,212 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,212 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,311 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:42,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:42,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:42,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:42,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,342 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:42,342 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,342 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-27 16:53:42,342 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-27 16:53:42,343 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,343 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,359 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:42,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-27 16:53:42,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,361 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-27 16:53:42,361 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:42,362 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:42,362 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-27 16:53:42,363 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. started... 2023-04-27 16:53:42,363 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,363 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,363 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:42,369 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,379 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,380 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,379 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,385 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/a9f1fc9349954ff890f47911a53ddbcf 2023-04-27 16:53:42,407 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c 2023-04-27 16:53:42,415 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/a9f1fc9349954ff890f47911a53ddbcf as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf 2023-04-27 16:53:42,421 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:42,423 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c 2023-04-27 16:53:42,430 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:42,431 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 68ms, sequenceid=71, compaction requested=true 2023-04-27 16:53:42,431 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:42,431 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,431 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:42,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:42,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-27 16:53:42,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-27 16:53:42,442 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,442 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,442 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,443 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-27 16:53:42,445 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-27 16:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a3f0ef1[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:42,445 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:42,453 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:42,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,454 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-27 16:53:42,454 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-27 16:53:42,454 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:42,463 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,463 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,463 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,463 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,463 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,463 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,464 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,474 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-27 16:53:42,474 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,474 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-27 16:53:42,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:42,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:53:42,478 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-27 16:53:42,478 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:42,478 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:42,478 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,478 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,485 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:42,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,485 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:42,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,485 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,486 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:42,487 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,487 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,488 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,495 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,495 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-27 16:53:42,496 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:42,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,495 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:42,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:42,496 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:42,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,497 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-27 16:53:42,597 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:42,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:42,621 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,627 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:42,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:42,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:42,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:42,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, UNASSIGN}] 2023-04-27 16:53:42,638 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, UNASSIGN 2023-04-27 16:53:42,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6ce00783d83d84715d1e04b95abdb46b, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:42,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614422640"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614422640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614422640"}]},"ts":"1682614422640"} 2023-04-27 16:53:42,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 6ce00783d83d84715d1e04b95abdb46b, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:42,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 6ce00783d83d84715d1e04b95abdb46b, disabling compactions & flushes 2023-04-27 16:53:42,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. after waiting 0 ms 2023-04-27 16:53:42,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 6ce00783d83d84715d1e04b95abdb46b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:53:42,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/555a813dc0174e8c81920237659fb6ec 2023-04-27 16:53:42,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/7d509420d09a48e5b218bc3a75dc8c9f 2023-04-27 16:53:42,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf1/555a813dc0174e8c81920237659fb6ec as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec 2023-04-27 16:53:42,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:42,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.tmp/cf2/7d509420d09a48e5b218bc3a75dc8c9f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f 2023-04-27 16:53:42,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:42,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6ce00783d83d84715d1e04b95abdb46b in 66ms, sequenceid=82, compaction requested=true 2023-04-27 16:53:42,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:53:42,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:42,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. 2023-04-27 16:53:42,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 6ce00783d83d84715d1e04b95abdb46b: 2023-04-27 16:53:42,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,893 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=6ce00783d83d84715d1e04b95abdb46b, regionState=CLOSED 2023-04-27 16:53:42,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614422893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614422893"}]},"ts":"1682614422893"} 2023-04-27 16:53:42,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-04-27 16:53:42,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 6ce00783d83d84715d1e04b95abdb46b, server=jenkins-hbase12.apache.org,46011,1682614416374 in 254 msec 2023-04-27 16:53:42,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-04-27 16:53:42,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ce00783d83d84715d1e04b95abdb46b, UNASSIGN in 262 msec 2023-04-27 16:53:42,919 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=6ce00783d83d84715d1e04b95abdb46b, threads=14 2023-04-27 16:53:42,920 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,920 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,921 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,921 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,921 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,921 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,922 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,922 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,922 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,922 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,922 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,923 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,923 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,924 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:42,949 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962, top=true 2023-04-27 16:53:42,965 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c, top=true 2023-04-27 16:53:42,965 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec, top=true 2023-04-27 16:53:42,974 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f, top=true 2023-04-27 16:53:43,021 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c, top=true 2023-04-27 16:53:43,025 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf, top=true 2023-04-27 16:53:43,066 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962 for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,066 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,068 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,080 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,088 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,101 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,090 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,089 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,102 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,102 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,105 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/.splits/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec for child: 7c40e5afe141126a7424fd4ac3bbbe94, parent: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,111 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,145 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,162 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,172 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,172 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,188 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,188 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,189 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,488 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0 for region: 6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:43,488 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 6ce00783d83d84715d1e04b95abdb46b Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:53:43,551 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:43,555 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:43,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1682614423558"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1682614423558"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1682614423558"}]},"ts":"1682614423558"} 2023-04-27 16:53:43,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614423558"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614423558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614423558"}]},"ts":"1682614423558"} 2023-04-27 16:53:43,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614423558"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614423558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614423558"}]},"ts":"1682614423558"} 2023-04-27 16:53:43,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, ASSIGN}] 2023-04-27 16:53:43,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, ASSIGN 2023-04-27 16:53:43,607 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:43,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, ASSIGN 2023-04-27 16:53:43,609 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:43,758 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:53:43,759 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7885756366b98522751324208da7edc7, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:43,759 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614423759"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614423759"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614423759"}]},"ts":"1682614423759"} 2023-04-27 16:53:43,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 7885756366b98522751324208da7edc7, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:43,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7c40e5afe141126a7424fd4ac3bbbe94, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:43,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614423766"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614423766"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614423766"}]},"ts":"1682614423766"} 2023-04-27 16:53:43,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 7c40e5afe141126a7424fd4ac3bbbe94, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:43,883 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:53:43,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:43,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7c40e5afe141126a7424fd4ac3bbbe94, NAME => 'testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.', STARTKEY => '26', ENDKEY => ''} 2023-04-27 16:53:43,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:43,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:43,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:43,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:43,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:43,923 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:43,927 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1 2023-04-27 16:53:43,927 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1 2023-04-27 16:53:43,928 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c40e5afe141126a7424fd4ac3bbbe94 columnFamilyName cf1 2023-04-27 16:53:43,958 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-04-27 16:53:43,959 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-04-27 16:53:43,978 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-top 2023-04-27 16:53:43,986 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-top 2023-04-27 16:53:43,994 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-top 2023-04-27 16:53:44,002 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-top 2023-04-27 16:53:44,009 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec 2023-04-27 16:53:44,016 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c 2023-04-27 16:53:44,022 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf 2023-04-27 16:53:44,023 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(310): Store=7c40e5afe141126a7424fd4ac3bbbe94/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:44,023 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,024 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2 2023-04-27 16:53:44,024 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2 2023-04-27 16:53:44,025 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c40e5afe141126a7424fd4ac3bbbe94 columnFamilyName cf2 2023-04-27 16:53:44,041 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-top 2023-04-27 16:53:44,047 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-top 2023-04-27 16:53:44,053 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-top 2023-04-27 16:53:44,059 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-top 2023-04-27 16:53:44,065 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962 2023-04-27 16:53:44,071 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f 2023-04-27 16:53:44,077 DEBUG [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c 2023-04-27 16:53:44,077 INFO [StoreOpener-7c40e5afe141126a7424fd4ac3bbbe94-1] regionserver.HStore(310): Store=7c40e5afe141126a7424fd4ac3bbbe94/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:44,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:44,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 7c40e5afe141126a7424fd4ac3bbbe94; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70371499, jitterRate=0.04861705005168915}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:44,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 7c40e5afe141126a7424fd4ac3bbbe94: 2023-04-27 16:53:44,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94., pid=18, masterSystemTime=1682614423915 2023-04-27 16:53:44,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:53:44,091 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:44,091 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:44,093 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,093 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7885756366b98522751324208da7edc7, NAME => 'testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.', STARTKEY => '', ENDKEY => '26'} 2023-04-27 16:53:44,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:44,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:44,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,093 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): 7c40e5afe141126a7424fd4ac3bbbe94/cf1 is initiating minor compaction (all files) 2023-04-27 16:53:44,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7c40e5afe141126a7424fd4ac3bbbe94, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:44,095 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7c40e5afe141126a7424fd4ac3bbbe94/cf1 in testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424093"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614424093"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614424093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614424093"}]},"ts":"1682614424093"} 2023-04-27 16:53:44,093 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): 7c40e5afe141126a7424fd4ac3bbbe94/cf2 is initiating minor compaction (all files) 2023-04-27 16:53:44,095 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/.tmp, totalSize=34.9 K 2023-04-27 16:53:44,096 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7c40e5afe141126a7424fd4ac3bbbe94/cf2 in testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,097 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/.tmp, totalSize=34.9 K 2023-04-27 16:53:44,098 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614420490 2023-04-27 16:53:44,098 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,099 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614420490 2023-04-27 16:53:44,100 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614420997 2023-04-27 16:53:44,101 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614420997 2023-04-27 16:53:44,102 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614421360 2023-04-27 16:53:44,102 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614421360 2023-04-27 16:53:44,102 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1 2023-04-27 16:53:44,102 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1 2023-04-27 16:53:44,103 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614421673 2023-04-27 16:53:44,103 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7885756366b98522751324208da7edc7 columnFamilyName cf1 2023-04-27 16:53:44,104 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614422000 2023-04-27 16:53:44,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-04-27 16:53:44,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 7c40e5afe141126a7424fd4ac3bbbe94, server=jenkins-hbase12.apache.org,46011,1682614416374 in 331 msec 2023-04-27 16:53:44,105 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614421673 2023-04-27 16:53:44,107 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614422314 2023-04-27 16:53:44,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, ASSIGN in 500 msec 2023-04-27 16:53:44,108 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614422000 2023-04-27 16:53:44,109 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614422599 2023-04-27 16:53:44,110 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614422314 2023-04-27 16:53:44,110 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614422599 2023-04-27 16:53:44,150 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7c40e5afe141126a7424fd4ac3bbbe94#cf1#compaction#14 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:44,150 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-bottom 2023-04-27 16:53:44,150 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 7c40e5afe141126a7424fd4ac3bbbe94#cf2#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:44,159 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-bottom 2023-04-27 16:53:44,166 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-bottom 2023-04-27 16:53:44,172 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-bottom 2023-04-27 16:53:44,173 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(310): Store=7885756366b98522751324208da7edc7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:44,173 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,174 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2 2023-04-27 16:53:44,175 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2 2023-04-27 16:53:44,175 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7885756366b98522751324208da7edc7 columnFamilyName cf2 2023-04-27 16:53:44,184 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/.tmp/cf1/0e9167ac02d04aafa8d260dd4af5f482 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/0e9167ac02d04aafa8d260dd4af5f482 2023-04-27 16:53:44,185 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/.tmp/cf2/632608cccf404d5e9204bbe3c6bca018 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/632608cccf404d5e9204bbe3c6bca018 2023-04-27 16:53:44,195 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-bottom 2023-04-27 16:53:44,202 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7c40e5afe141126a7424fd4ac3bbbe94/cf1 of 7c40e5afe141126a7424fd4ac3bbbe94 into 0e9167ac02d04aafa8d260dd4af5f482(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:44,202 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7c40e5afe141126a7424fd4ac3bbbe94/cf2 of 7c40e5afe141126a7424fd4ac3bbbe94 into 632608cccf404d5e9204bbe3c6bca018(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:44,205 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7c40e5afe141126a7424fd4ac3bbbe94: 2023-04-27 16:53:44,205 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7c40e5afe141126a7424fd4ac3bbbe94: 2023-04-27 16:53:44,205 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94., storeName=7c40e5afe141126a7424fd4ac3bbbe94/cf1, priority=9, startTime=1682614424089; duration=0sec 2023-04-27 16:53:44,205 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94., storeName=7c40e5afe141126a7424fd4ac3bbbe94/cf2, priority=9, startTime=1682614424090; duration=0sec 2023-04-27 16:53:44,205 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-bottom 2023-04-27 16:53:44,206 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,206 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,212 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-bottom 2023-04-27 16:53:44,218 DEBUG [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-bottom 2023-04-27 16:53:44,218 INFO [StoreOpener-7885756366b98522751324208da7edc7-1] regionserver.HStore(310): Store=7885756366b98522751324208da7edc7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:44,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7 2023-04-27 16:53:44,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7 2023-04-27 16:53:44,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:44,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 7885756366b98522751324208da7edc7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59108057, jitterRate=-0.11922131478786469}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:44,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 7885756366b98522751324208da7edc7: 2023-04-27 16:53:44,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7., pid=17, masterSystemTime=1682614423915 2023-04-27 16:53:44,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,227 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-27 16:53:44,227 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-27 16:53:44,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:53:44,227 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,227 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,227 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): 7885756366b98522751324208da7edc7/cf1 is initiating minor compaction (all files) 2023-04-27 16:53:44,227 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): 7885756366b98522751324208da7edc7/cf2 is initiating minor compaction (all files) 2023-04-27 16:53:44,228 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7885756366b98522751324208da7edc7/cf1 in testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,228 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7885756366b98522751324208da7edc7/cf2 in testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,228 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/.tmp, totalSize=19.9 K 2023-04-27 16:53:44,228 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/.tmp, totalSize=19.9 K 2023-04-27 16:53:44,229 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614420490 2023-04-27 16:53:44,229 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614420490 2023-04-27 16:53:44,230 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614420997 2023-04-27 16:53:44,230 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614420997 2023-04-27 16:53:44,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,231 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614421360 2023-04-27 16:53:44,231 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614421360 2023-04-27 16:53:44,231 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7885756366b98522751324208da7edc7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:44,231 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424231"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614424231"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614424231"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614424231"}]},"ts":"1682614424231"} 2023-04-27 16:53:44,231 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614421673 2023-04-27 16:53:44,231 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614421673 2023-04-27 16:53:44,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-04-27 16:53:44,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 7885756366b98522751324208da7edc7, server=jenkins-hbase12.apache.org,46011,1682614416374 in 472 msec 2023-04-27 16:53:44,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-04-27 16:53:44,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, ASSIGN in 632 msec 2023-04-27 16:53:44,244 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 7885756366b98522751324208da7edc7#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:44,244 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7885756366b98522751324208da7edc7#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:44,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 in 1.6200 sec 2023-04-27 16:53:44,262 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/.tmp/cf1/3a3035d242a84478bbc85b87b164b95b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/3a3035d242a84478bbc85b87b164b95b 2023-04-27 16:53:44,263 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/.tmp/cf2/80843c9223de40d3b639527303d77715 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/80843c9223de40d3b639527303d77715 2023-04-27 16:53:44,269 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7885756366b98522751324208da7edc7/cf1 of 7885756366b98522751324208da7edc7 into 3a3035d242a84478bbc85b87b164b95b(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:44,269 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7885756366b98522751324208da7edc7: 2023-04-27 16:53:44,269 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7., storeName=7885756366b98522751324208da7edc7/cf1, priority=12, startTime=1682614424226; duration=0sec 2023-04-27 16:53:44,269 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,270 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7885756366b98522751324208da7edc7/cf2 of 7885756366b98522751324208da7edc7 into 80843c9223de40d3b639527303d77715(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:44,270 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7885756366b98522751324208da7edc7: 2023-04-27 16:53:44,270 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7., storeName=7885756366b98522751324208da7edc7/cf2, priority=12, startTime=1682614424227; duration=0sec 2023-04-27 16:53:44,270 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:44,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46011] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:44546 deadline: 1682614484382, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b. is not online on jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:44,515 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-04-27 16:53:44,530 INFO [Listener at localhost.localdomain/45365] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-27 16:53:44,534 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-04-27 16:53:44,570 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:44,580 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-04-27 16:53:44,586 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitTableRegionUnevenDaughter 2023-04-27 16:53:44,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:44,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614424600"}]},"ts":"1682614424600"} 2023-04-27 16:53:44,603 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-04-27 16:53:44,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-27 16:53:44,663 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-04-27 16:53:44,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, UNASSIGN}] 2023-04-27 16:53:44,667 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, UNASSIGN 2023-04-27 16:53:44,668 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7c40e5afe141126a7424fd4ac3bbbe94, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:44,668 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424668"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614424668"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614424668"}]},"ts":"1682614424668"} 2023-04-27 16:53:44,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7c40e5afe141126a7424fd4ac3bbbe94, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:44,674 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, UNASSIGN 2023-04-27 16:53:44,675 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7885756366b98522751324208da7edc7, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:44,675 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424674"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614424674"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614424674"}]},"ts":"1682614424674"} 2023-04-27 16:53:44,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 7885756366b98522751324208da7edc7, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:44,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-27 16:53:44,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 7885756366b98522751324208da7edc7, disabling compactions & flushes 2023-04-27 16:53:44,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. after waiting 0 ms 2023-04-27 16:53:44,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,831 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-bottom] to archive 2023-04-27 16:53:44,832 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:44,838 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,840 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,842 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,844 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,866 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-bottom] to archive 2023-04-27 16:53:44,867 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:44,870 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,872 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,873 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,875 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:53:44,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:44,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7. 2023-04-27 16:53:44,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 7885756366b98522751324208da7edc7: 2023-04-27 16:53:44,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7885756366b98522751324208da7edc7 2023-04-27 16:53:44,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 7c40e5afe141126a7424fd4ac3bbbe94, disabling compactions & flushes 2023-04-27 16:53:44,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. after waiting 0 ms 2023-04-27 16:53:44,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7885756366b98522751324208da7edc7, regionState=CLOSED 2023-04-27 16:53:44,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614424884"}]},"ts":"1682614424884"} 2023-04-27 16:53:44,886 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec] to archive 2023-04-27 16:53:44,887 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:44,890 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/993dc1d20d854fe2a49161b692859199.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-04-27 16:53:44,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 7885756366b98522751324208da7edc7, server=jenkins-hbase12.apache.org,46011,1682614416374 in 210 msec 2023-04-27 16:53:44,894 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/ea1e1d20ae284a0c9a98b31332d18b0d.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7885756366b98522751324208da7edc7, UNASSIGN in 226 msec 2023-04-27 16:53:44,896 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/7d3b35f3edc5499ebfc2eccb99fc9177.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,898 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/6fbe84df577a449ebb8c5cb11eb79679.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,899 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-570ffb34590f4add800e3e49db77a93c 2023-04-27 16:53:44,901 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-a9f1fc9349954ff890f47911a53ddbcf 2023-04-27 16:53:44,902 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-555a813dc0174e8c81920237659fb6ec 2023-04-27 16:53:44,904 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f] to archive 2023-04-27 16:53:44,905 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:44,908 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,909 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/1fa73873fd294cffaa072d4fd8dd104d.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,911 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/973140fb6efb41738e96cd3f398a06a0.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,912 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/69a08e6a329d488ba0aa6c07c329d4c0.6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:44,914 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-0d732736c3814019816c6a04a8e9f962 2023-04-27 16:53:44,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-27 16:53:44,915 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-ecde0c7e671d435a8fd47b5ad88f0d6c 2023-04-27 16:53:44,917 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/testSplitTableRegionUnevenDaughter=6ce00783d83d84715d1e04b95abdb46b-7d509420d09a48e5b218bc3a75dc8c9f 2023-04-27 16:53:44,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:53:44,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:44,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94. 2023-04-27 16:53:44,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 7c40e5afe141126a7424fd4ac3bbbe94: 2023-04-27 16:53:44,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:44,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7c40e5afe141126a7424fd4ac3bbbe94, regionState=CLOSED 2023-04-27 16:53:44,927 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1682614424926"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614424926"}]},"ts":"1682614424926"} 2023-04-27 16:53:44,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-04-27 16:53:44,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7c40e5afe141126a7424fd4ac3bbbe94, server=jenkins-hbase12.apache.org,46011,1682614416374 in 259 msec 2023-04-27 16:53:44,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-04-27 16:53:44,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7c40e5afe141126a7424fd4ac3bbbe94, UNASSIGN in 267 msec 2023-04-27 16:53:44,937 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614424937"}]},"ts":"1682614424937"} 2023-04-27 16:53:44,939 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-04-27 16:53:44,989 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-04-27 16:53:44,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 403 msec 2023-04-27 16:53:45,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-27 16:53:45,217 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-04-27 16:53:45,221 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,229 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,230 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-27 16:53:45,230 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-04-27 16:53:45,231 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:45,231 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-04-27 16:53:45,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-27 16:53:45,239 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:45,239 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7 2023-04-27 16:53:45,239 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:45,243 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/recovered.edits] 2023-04-27 16:53:45,243 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/recovered.edits] 2023-04-27 16:53:45,243 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/recovered.edits] 2023-04-27 16:53:45,247 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/0e9167ac02d04aafa8d260dd4af5f482 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf1/0e9167ac02d04aafa8d260dd4af5f482 2023-04-27 16:53:45,247 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/555a813dc0174e8c81920237659fb6ec 2023-04-27 16:53:45,247 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/3a3035d242a84478bbc85b87b164b95b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf1/3a3035d242a84478bbc85b87b164b95b 2023-04-27 16:53:45,248 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/570ffb34590f4add800e3e49db77a93c 2023-04-27 16:53:45,250 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/80843c9223de40d3b639527303d77715 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/cf2/80843c9223de40d3b639527303d77715 2023-04-27 16:53:45,250 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/632608cccf404d5e9204bbe3c6bca018 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/cf2/632608cccf404d5e9204bbe3c6bca018 2023-04-27 16:53:45,250 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/6fbe84df577a449ebb8c5cb11eb79679 2023-04-27 16:53:45,252 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/7d3b35f3edc5499ebfc2eccb99fc9177 2023-04-27 16:53:45,254 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/993dc1d20d854fe2a49161b692859199 2023-04-27 16:53:45,254 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94/recovered.edits/90.seqid 2023-04-27 16:53:45,254 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7/recovered.edits/90.seqid 2023-04-27 16:53:45,255 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:45,255 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/7885756366b98522751324208da7edc7 2023-04-27 16:53:45,256 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/a9f1fc9349954ff890f47911a53ddbcf 2023-04-27 16:53:45,258 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf1/ea1e1d20ae284a0c9a98b31332d18b0d 2023-04-27 16:53:45,260 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/0d732736c3814019816c6a04a8e9f962 2023-04-27 16:53:45,262 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/1fa73873fd294cffaa072d4fd8dd104d 2023-04-27 16:53:45,263 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/69a08e6a329d488ba0aa6c07c329d4c0 2023-04-27 16:53:45,265 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/7d509420d09a48e5b218bc3a75dc8c9f 2023-04-27 16:53:45,266 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/973140fb6efb41738e96cd3f398a06a0 2023-04-27 16:53:45,268 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/dbbd24263c2e4d5eabe7310d2e33ad2c 2023-04-27 16:53:45,269 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/cf2/ecde0c7e671d435a8fd47b5ad88f0d6c 2023-04-27 16:53:45,273 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b/recovered.edits/85.seqid 2023-04-27 16:53:45,273 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ce00783d83d84715d1e04b95abdb46b 2023-04-27 16:53:45,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-04-27 16:53:45,277 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,296 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-04-27 16:53:45,333 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-04-27 16:53:45,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-27 16:53:45,335 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,335 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-04-27 16:53:45,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614425335"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:45,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614425335"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:45,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614425335"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:45,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:53:45,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ce00783d83d84715d1e04b95abdb46b, NAME => 'testSplitTableRegionUnevenDaughter,,1682614419732.6ce00783d83d84715d1e04b95abdb46b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 7885756366b98522751324208da7edc7, NAME => 'testSplitTableRegionUnevenDaughter,,1682614422621.7885756366b98522751324208da7edc7.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7c40e5afe141126a7424fd4ac3bbbe94, NAME => 'testSplitTableRegionUnevenDaughter,26,1682614422621.7c40e5afe141126a7424fd4ac3bbbe94.', STARTKEY => '26', ENDKEY => ''}] 2023-04-27 16:53:45,339 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-04-27 16:53:45,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614425339"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:45,342 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-04-27 16:53:45,401 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:45,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 179 msec 2023-04-27 16:53:45,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-27 16:53:45,536 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-04-27 16:53:45,553 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=390 (was 359) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37944 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:50758 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:50816 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37892 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:33278 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase12:46011-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:50700 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39236 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39482 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39478 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:38002 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37858 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:50850 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39510 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:50778 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:44626 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39434 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=820 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=411 (was 403) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=11932 (was 12022) 2023-04-27 16:53:45,568 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=390, OpenFileDescriptor=820, MaxFileDescriptor=60000, SystemLoadAverage=411, ProcessCount=174, AvailableMemoryMB=11931 2023-04-27 16:53:45,571 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:45,611 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:53:45,612 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:53:45,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:45,615 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:53:45,616 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:53:45,618 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:45,619 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc empty. 2023-04-27 16:53:45,620 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:45,620 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-04-27 16:53:45,632 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:45,633 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 985ef773fc4ca76ee5d569ef4a9b03cc, NAME => 'testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 985ef773fc4ca76ee5d569ef4a9b03cc, disabling compactions & flushes 2023-04-27 16:53:45,642 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. after waiting 0 ms 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:45,642 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:45,642 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:45,645 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:53:45,646 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614425646"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614425646"}]},"ts":"1682614425646"} 2023-04-27 16:53:45,648 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:53:45,649 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:53:45,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614425649"}]},"ts":"1682614425649"} 2023-04-27 16:53:45,651 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-04-27 16:53:45,684 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:53:45,685 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:53:45,685 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:53:45,685 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:53:45,685 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:53:45,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, ASSIGN}] 2023-04-27 16:53:45,688 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, ASSIGN 2023-04-27 16:53:45,689 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:45,840 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:53:45,841 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:45,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614425841"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614425841"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614425841"}]},"ts":"1682614425841"} 2023-04-27 16:53:45,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:46,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 985ef773fc4ca76ee5d569ef4a9b03cc, NAME => 'testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:46,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:46,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:46,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,002 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,003 DEBUG [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1 2023-04-27 16:53:46,004 DEBUG [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1 2023-04-27 16:53:46,004 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 985ef773fc4ca76ee5d569ef4a9b03cc columnFamilyName cf1 2023-04-27 16:53:46,005 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] regionserver.HStore(310): Store=985ef773fc4ca76ee5d569ef4a9b03cc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:46,005 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,007 DEBUG [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2 2023-04-27 16:53:46,007 DEBUG [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2 2023-04-27 16:53:46,008 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 985ef773fc4ca76ee5d569ef4a9b03cc columnFamilyName cf2 2023-04-27 16:53:46,009 INFO [StoreOpener-985ef773fc4ca76ee5d569ef4a9b03cc-1] regionserver.HStore(310): Store=985ef773fc4ca76ee5d569ef4a9b03cc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:46,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:46,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:46,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:46,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 985ef773fc4ca76ee5d569ef4a9b03cc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73051997, jitterRate=0.0885595828294754}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:46,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:46,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc., pid=27, masterSystemTime=1682614425996 2023-04-27 16:53:46,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614426021"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614426021"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614426021"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614426021"}]},"ts":"1682614426021"} 2023-04-27 16:53:46,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-04-27 16:53:46,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 in 180 msec 2023-04-27 16:53:46,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-04-27 16:53:46,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, ASSIGN in 342 msec 2023-04-27 16:53:46,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:53:46,033 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614426032"}]},"ts":"1682614426032"} 2023-04-27 16:53:46,034 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-04-27 16:53:46,085 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:53:46,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 472 msec 2023-04-27 16:53:46,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:46,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:46,131 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:46,131 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,132 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:46,132 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:46,132 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,132 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,137 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,137 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,139 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,139 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,139 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,140 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,140 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,140 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,140 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,140 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,141 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,141 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,141 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,141 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,141 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,142 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,142 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:46,142 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,142 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:46,142 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:46,143 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:53:46,147 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,158 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,158 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,158 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,158 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,158 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,159 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/5c18ea1cdd004dcaba690b45d41feda2 2023-04-27 16:53:46,179 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/a72b6106ab344b2ab73aaee2d427fb8f 2023-04-27 16:53:46,184 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/5c18ea1cdd004dcaba690b45d41feda2 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 2023-04-27 16:53:46,190 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:46,191 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/a72b6106ab344b2ab73aaee2d427fb8f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f 2023-04-27 16:53:46,197 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:46,198 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 55ms, sequenceid=6, compaction requested=false 2023-04-27 16:53:46,198 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-04-27 16:53:46,199 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:46,199 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,199 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:46,199 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:46,199 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,199 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,199 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,210 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,210 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,211 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,211 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76be998a[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:46,213 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:46,213 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,221 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,221 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:46,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:46,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:46,231 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,231 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,231 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,231 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,232 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,232 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,242 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,242 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,242 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:46,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:46,247 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:46,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:46,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:46,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:46,247 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:46,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,253 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,254 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,256 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,263 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,263 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:46,264 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,264 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:46,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:46,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,265 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,266 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,266 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,365 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:46,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:46,389 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:46,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:46,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:46,393 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:46,393 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,393 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:46,393 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:46,394 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,394 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,400 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,400 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,400 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,400 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,402 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,403 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,404 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:46,404 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:46,404 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,404 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:46,404 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:46,410 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,410 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,411 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,411 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,411 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,411 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,419 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/0c7ceab6e9a5438ca482a90c20f3a405 2023-04-27 16:53:46,440 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 2023-04-27 16:53:46,446 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/0c7ceab6e9a5438ca482a90c20f3a405 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 2023-04-27 16:53:46,451 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:46,452 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 2023-04-27 16:53:46,458 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:46,459 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 55ms, sequenceid=19, compaction requested=false 2023-04-27 16:53:46,459 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:46,460 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,460 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:46,460 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:46,460 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,460 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,460 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:46,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66437ac1[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:46,475 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:46,475 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,484 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:46,484 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,485 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:46,555 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,556 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,556 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,568 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,568 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,568 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,568 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,568 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,569 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:46,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:46,576 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:46,576 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:46,576 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:46,576 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:46,576 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:46,588 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,588 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,588 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,588 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,589 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,589 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,590 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,590 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,591 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,592 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,592 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:46,600 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,600 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:46,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,600 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,601 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,602 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,701 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:46,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:46,724 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:46,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:46,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:46,726 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:46,726 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,727 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:46,727 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:46,728 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,728 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,737 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,737 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,737 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:46,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:46,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:46,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,741 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,741 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,741 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,741 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,742 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,743 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:46,743 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,743 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,743 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:46,743 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:46,743 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,743 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:46,744 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:46,747 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,758 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/692281cca0fd407a97afd79588c9477d 2023-04-27 16:53:46,760 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,760 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,761 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,761 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,761 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,776 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d 2023-04-27 16:53:46,783 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/692281cca0fd407a97afd79588c9477d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d 2023-04-27 16:53:46,789 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:46,790 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d 2023-04-27 16:53:46,796 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:46,798 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 54ms, sequenceid=32, compaction requested=true 2023-04-27 16:53:46,798 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:46,798 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:46,798 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:46,798 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:46,798 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,798 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:46,798 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:46,810 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,811 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,811 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:46,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f49331e[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:46,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:46,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,821 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,821 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:46,822 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:46,831 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,831 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,831 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,832 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,832 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,832 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:46,842 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,842 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:46,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:46,845 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:46,845 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:46,845 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:46,845 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:46,845 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:46,852 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,852 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,852 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,852 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,852 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,853 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:46,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,854 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,854 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,856 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,856 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:46,857 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,864 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:46,864 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:46,864 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:46,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:46,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:46,863 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:46,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,864 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:46,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:46,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:46,865 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,965 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:46,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:46,989 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:46,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:46,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:46,992 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:46,992 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:46,992 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:46,992 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:46,993 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:46,993 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,032 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,032 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,032 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:47,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,034 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,034 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,034 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,035 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:47,038 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,038 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:47,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,038 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:47,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,038 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:47,052 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,054 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/fc4c509131e749d3b32efb5c6d9085b2 2023-04-27 16:53:47,063 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,063 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,063 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,063 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,063 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,075 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/cb6b28687db04cc8b0f29dc0eacd9740 2023-04-27 16:53:47,081 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/fc4c509131e749d3b32efb5c6d9085b2 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 2023-04-27 16:53:47,087 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:47,088 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/cb6b28687db04cc8b0f29dc0eacd9740 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 2023-04-27 16:53:47,095 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:47,096 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 58ms, sequenceid=45, compaction requested=true 2023-04-27 16:53:47,097 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:47,097 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,097 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:47,097 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:47,097 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,097 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,097 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,105 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,106 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,106 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d23f680[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:47,108 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:47,108 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,116 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,116 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:47,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:47,116 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:47,126 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,126 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,126 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,126 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,137 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,147 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,147 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,147 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:47,150 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:47,151 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:47,151 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:47,151 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:47,151 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,159 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,159 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,160 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,161 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,161 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,161 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,169 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,168 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,169 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:47,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,169 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,169 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,170 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,270 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:47,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:47,293 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:47,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:47,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:47,295 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:47,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:47,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:47,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,305 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,305 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,306 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,307 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,307 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,307 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,308 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:47,315 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,315 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,316 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,316 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,316 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,316 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,323 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/06813a281dbb4c718f9306c2cfbd29a6 2023-04-27 16:53:47,340 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/1e05060b346b4d2e8b32b884d487a0ee 2023-04-27 16:53:47,347 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/06813a281dbb4c718f9306c2cfbd29a6 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 2023-04-27 16:53:47,353 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:47,354 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/1e05060b346b4d2e8b32b884d487a0ee as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee 2023-04-27 16:53:47,359 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:47,360 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 53ms, sequenceid=58, compaction requested=true 2023-04-27 16:53:47,361 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:47,361 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,361 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:47,361 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:47,361 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,361 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,361 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,368 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,369 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,369 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:47,371 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:47,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fd34267[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:47,371 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,379 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:47,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:47,379 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,380 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,380 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:47,389 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,389 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,389 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,400 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,400 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,400 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,400 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,410 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,410 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,410 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:47,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:47,414 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:47,414 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:47,414 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:47,414 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:47,414 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,447 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,446 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,447 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,447 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,449 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,449 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,449 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,472 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,473 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,472 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:47,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,574 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:47,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:47,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:47,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:47,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-27 16:53:47,598 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-27 16:53:47,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-27 16:53:47,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-27 16:53:47,599 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,599 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,610 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,610 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,610 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,610 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,611 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,611 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:47,612 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,612 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,612 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:47,613 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:47,613 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,613 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. started... 2023-04-27 16:53:47,613 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:47,621 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,627 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d 2023-04-27 16:53:47,631 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,631 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,631 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,631 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,631 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,642 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/4a585a41e54f4be69a1d9724e2e795bf 2023-04-27 16:53:47,648 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d 2023-04-27 16:53:47,654 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:47,655 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/4a585a41e54f4be69a1d9724e2e795bf as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf 2023-04-27 16:53:47,661 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:47,662 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 49ms, sequenceid=71, compaction requested=true 2023-04-27 16:53:47,663 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:47,663 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:47,663 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:47,663 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:47,663 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,663 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-27 16:53:47,663 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-27 16:53:47,673 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,673 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,674 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,674 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-27 16:53:47,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23bba2e[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:47,676 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-27 16:53:47,676 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,684 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:47,685 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:47,694 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,695 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,695 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,695 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,695 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,695 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,705 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-27 16:53:47,705 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,705 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,705 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-27 16:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-27 16:53:47,709 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-27 16:53:47,709 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:47,709 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:47,709 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-27 16:53:47,709 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,716 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,717 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,717 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:47,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,718 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,719 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:47,719 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-27 16:53:47,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:47,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:47,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:47,726 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-27 16:53:47,727 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:47,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:47,726 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:47,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:47,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-27 16:53:47,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:47,727 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,728 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:47,728 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:47,728 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:47,728 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-27 16:53:47,728 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-27 16:53:47,828 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:47,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:48,094 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-27 16:53:48,096 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:48,097 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,098 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,098 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,098 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,099 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:48,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:48,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:48,348 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,349 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:48,349 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:48,349 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:48,352 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-04-27 16:53:48,353 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:48,353 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:48,353 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:48,353 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:48,354 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:48,357 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-27 16:53:48,359 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:48,360 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:48,361 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:48,361 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:48,361 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,362 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:48,363 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,363 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,364 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:48,365 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,365 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:48,365 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,365 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,365 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,366 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-04-27 16:53:48,367 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:48,368 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:48,369 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614458369 2023-04-27 16:53:48,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,371 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:48,375 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,376 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,376 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:48,376 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614488376 2023-04-27 16:53:48,376 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614548376 2023-04-27 16:53:48,376 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-27 16:53:48,376 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:48,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:48,379 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:48,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:48,626 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,626 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:48,627 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:48,627 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:48,628 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:48,628 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:48,628 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:48,628 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:48,628 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:48,629 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:48,630 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-27 16:53:48,631 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:48,631 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:48,631 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:48,631 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:48,632 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,632 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:48,632 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,632 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,633 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:48,633 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,633 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:48,633 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,633 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,633 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,634 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:48,634 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:48,635 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:48,635 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614458635 2023-04-27 16:53:48,637 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,637 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:48,640 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,641 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,641 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:48,641 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614488641 2023-04-27 16:53:48,641 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614548641 2023-04-27 16:53:48,641 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:53:48,641 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:48,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN}] 2023-04-27 16:53:48,644 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:48,644 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:48,644 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:48,891 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,891 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:48,891 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:48,892 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:48,892 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:48,892 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:48,892 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:48,892 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:48,892 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:48,893 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:48,894 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-27 16:53:48,895 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:48,895 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:48,895 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:48,895 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:48,896 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,896 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:48,897 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,897 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:48,897 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:48,898 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,898 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:48,898 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,898 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,898 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,898 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:48,899 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:48,900 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:48,900 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614458900 2023-04-27 16:53:48,901 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:48,901 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:48,904 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,904 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:48,905 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:48,905 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614488905 2023-04-27 16:53:48,905 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614548905 2023-04-27 16:53:48,905 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:48,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN}] 2023-04-27 16:53:48,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:48,907 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:48,908 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:48,908 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614428908"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614428908"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614428908"}]},"ts":"1682614428908"} 2023-04-27 16:53:48,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:48,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:48,910 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:48,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:48,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:49,155 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,155 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:49,155 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:49,155 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:49,156 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:49,157 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:49,157 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:49,157 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:49,157 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:49,158 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:49,159 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:49,159 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:53:49,159 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:49,160 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:49,160 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:49,160 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:49,161 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,161 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:49,161 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,161 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,162 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:49,163 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,163 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,163 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,163 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,163 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,163 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,163 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,163 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,163 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:49,164 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:49,165 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:49,165 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc to restore RIT 2023-04-27 16:53:49,165 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614459165 2023-04-27 16:53:49,166 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:49,166 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,169 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,169 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,169 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:49,169 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614489169 2023-04-27 16:53:49,170 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614549170 2023-04-27 16:53:49,170 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:49,170 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,170 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614429170"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614429170"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614429170"}]},"ts":"1682614429170"} 2023-04-27 16:53:49,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:49,173 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:49,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 985ef773fc4ca76ee5d569ef4a9b03cc, disabling compactions & flushes 2023-04-27 16:53:49,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:49,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:49,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. after waiting 0 ms 2023-04-27 16:53:49,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:49,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 985ef773fc4ca76ee5d569ef4a9b03cc 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:53:49,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/97d490c5459244b0873555e792b72994 2023-04-27 16:53:49,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/0172ac632b42489390ae05ba57baca60 2023-04-27 16:53:49,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf1/97d490c5459244b0873555e792b72994 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 2023-04-27 16:53:49,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:49,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/.tmp/cf2/0172ac632b42489390ae05ba57baca60 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 2023-04-27 16:53:49,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:49,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 985ef773fc4ca76ee5d569ef4a9b03cc in 44ms, sequenceid=82, compaction requested=true 2023-04-27 16:53:49,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:53:49,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:49,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. 2023-04-27 16:53:49,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 985ef773fc4ca76ee5d569ef4a9b03cc: 2023-04-27 16:53:49,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,394 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED 2023-04-27 16:53:49,394 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614429394"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614429394"}]},"ts":"1682614429394"} 2023-04-27 16:53:49,395 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:49,396 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:49,396 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:49,420 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,420 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:49,420 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:49,420 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:49,420 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:49,421 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:49,421 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:49,421 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:49,421 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:49,422 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:49,423 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:49,423 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-27 16:53:49,424 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-04-27 16:53:49,424 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:53:49,424 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:49,424 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:49,425 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:49,425 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:49,425 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,426 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:49,426 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,426 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,426 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:49,427 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,427 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,427 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,428 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,428 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,428 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,428 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,428 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,428 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,428 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,428 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-27 16:53:49,429 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:49,430 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:49,430 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc to restore RIT 2023-04-27 16:53:49,432 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614459432 2023-04-27 16:53:49,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,433 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:49,436 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,437 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,437 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:49,437 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614489437 2023-04-27 16:53:49,437 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614549437 2023-04-27 16:53:49,437 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:53:49,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614429438"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614429438"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614429438"}]},"ts":"1682614429438"} 2023-04-27 16:53:49,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:49,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-04-27 16:53:49,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 in 268 msec 2023-04-27 16:53:49,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:49,444 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:49,445 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:49,688 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,688 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:49,688 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:49,688 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:49,688 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:49,689 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:49,689 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:49,689 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:49,689 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:49,690 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:49,691 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:49,691 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-27 16:53:49,691 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:49,691 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:49,692 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:49,692 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:49,693 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:49,693 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,693 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:49,694 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,694 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,694 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:49,695 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,695 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,695 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:49,695 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:49,695 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:49,696 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:49,697 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:49,697 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc to restore RIT 2023-04-27 16:53:49,697 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614459697 2023-04-27 16:53:49,698 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:49,698 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,701 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,701 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,701 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:49,701 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614489701 2023-04-27 16:53:49,701 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614549701 2023-04-27 16:53:49,701 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:49,702 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,704 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:49,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-04-27 16:53:49,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN in 799 msec 2023-04-27 16:53:49,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:49,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:49,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:49,952 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:49,952 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:49,953 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:49,953 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:49,954 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:49,955 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:49,956 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,956 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:49,957 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,957 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:49,957 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:49,958 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:49,958 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:49,959 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:49,960 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:49,960 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614459960 2023-04-27 16:53:49,962 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:49,962 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,965 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,965 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:49,965 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:49,965 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614489965 2023-04-27 16:53:49,966 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614549966 2023-04-27 16:53:49,966 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:49,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:49,972 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:49,983 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=985ef773fc4ca76ee5d569ef4a9b03cc, threads=14 2023-04-27 16:53:49,984 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,984 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,988 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,988 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,989 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,989 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,990 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,990 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,991 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,991 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,993 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,997 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,997 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:49,997 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,158 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,168 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,173 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,177 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,177 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,180 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,184 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,192 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,201 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,209 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,213 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,215 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,218 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,616 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,616 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 985ef773fc4ca76ee5d569ef4a9b03cc Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:53:50,654 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:50,654 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:50,654 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:50,716 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,716 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:50,716 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:50,716 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:50,717 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:50,717 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:50,717 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:50,717 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:50,717 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:50,718 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:50,719 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-27 16:53:50,719 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:50,719 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:50,720 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:50,720 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:50,721 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:50,721 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:50,722 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:50,722 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:50,722 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:50,723 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:50,723 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:50,724 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,725 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:50,725 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:50,725 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:50,725 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,725 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,725 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,725 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:50,725 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:50,725 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:53:50,726 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:50,727 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:50,727 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614460727 2023-04-27 16:53:50,729 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:50,729 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:50,732 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:50,732 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:50,732 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:50,732 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614490732 2023-04-27 16:53:50,732 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614550732 2023-04-27 16:53:50,733 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:50,743 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=985ef773fc4ca76ee5d569ef4a9b03cc, threads=14 2023-04-27 16:53:50,744 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,744 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,744 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,745 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,745 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,745 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,745 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,745 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,746 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,926 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,928 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,948 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,960 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,964 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,968 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,990 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:50,990 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,000 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,010 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,011 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,017 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,017 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,022 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 for region: 985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:51,022 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 985ef773fc4ca76ee5d569ef4a9b03cc Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:53:51,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:51,063 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:51,065 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:51,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:51,066 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:51,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:51,240 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:51,241 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:51,241 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:51,242 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:51,243 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-27 16:53:51,243 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:51,243 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:51,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:51,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:51,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:51,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:51,245 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,245 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:51,245 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,245 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,246 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:51,247 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,247 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,247 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:51,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:51,247 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:51,248 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:51,249 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:51,249 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614461249 2023-04-27 16:53:51,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,251 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:51,253 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:51,254 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:51,254 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:51,254 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614491254 2023-04-27 16:53:51,254 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614551254 2023-04-27 16:53:51,254 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:51,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:51,260 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:51,260 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:51,260 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:51,504 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:51,504 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:51,505 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:51,505 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:51,506 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:51,506 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:51,506 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:51,506 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:51,506 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:51,507 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:51,508 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-27 16:53:51,508 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:51,508 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:51,508 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:51,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:51,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:51,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:51,510 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,510 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:51,510 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,510 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:51,510 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:51,511 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,511 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN 2023-04-27 16:53:51,511 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:51,511 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:51,511 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,511 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,511 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,512 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=985ef773fc4ca76ee5d569ef4a9b03cc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:51,512 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 985ef773fc4ca76ee5d569ef4a9b03cc, server=jenkins-hbase12.apache.org,46011,1682614416374 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:51,512 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:51,512 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:51,513 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:51,513 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614461513 2023-04-27 16:53:51,513 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:51,515 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,517 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:51,518 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=985ef773fc4ca76ee5d569ef4a9b03cc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:51,518 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:51,518 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614491518 2023-04-27 16:53:51,518 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614551518 2023-04-27 16:53:51,518 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:53:51,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614431523"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1682614431523"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1682614431523"}]},"ts":"1682614431523"} 2023-04-27 16:53:51,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614431523"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614431523"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614431523"}]},"ts":"1682614431523"} 2023-04-27 16:53:51,524 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614431523"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614431523"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614431523"}]},"ts":"1682614431523"} 2023-04-27 16:53:51,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, ASSIGN}] 2023-04-27 16:53:51,531 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, ASSIGN 2023-04-27 16:53:51,533 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:51,534 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, ASSIGN 2023-04-27 16:53:51,535 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:53:51,683 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:53:51,685 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=537cccd40f1fb32014621fe16dc91849, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:51,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614431685"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614431685"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614431685"}]},"ts":"1682614431685"} 2023-04-27 16:53:51,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 537cccd40f1fb32014621fe16dc91849, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:51,691 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0c87014d6264f49d86caaf77094e33e7, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:51,691 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614431691"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614431691"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614431691"}]},"ts":"1682614431691"} 2023-04-27 16:53:51,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 0c87014d6264f49d86caaf77094e33e7, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:51,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0c87014d6264f49d86caaf77094e33e7, NAME => 'testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.', STARTKEY => '41', ENDKEY => ''} 2023-04-27 16:53:51,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,846 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,846 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1 2023-04-27 16:53:51,847 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1 2023-04-27 16:53:51,847 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c87014d6264f49d86caaf77094e33e7 columnFamilyName cf1 2023-04-27 16:53:51,865 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-top 2023-04-27 16:53:51,871 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-top 2023-04-27 16:53:51,877 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-top 2023-04-27 16:53:51,883 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-top 2023-04-27 16:53:51,890 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-top 2023-04-27 16:53:51,895 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-top 2023-04-27 16:53:51,901 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-top 2023-04-27 16:53:51,901 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(310): Store=0c87014d6264f49d86caaf77094e33e7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:51,901 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,903 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2 2023-04-27 16:53:51,903 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2 2023-04-27 16:53:51,903 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c87014d6264f49d86caaf77094e33e7 columnFamilyName cf2 2023-04-27 16:53:51,921 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-top 2023-04-27 16:53:51,926 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-top 2023-04-27 16:53:51,932 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-top 2023-04-27 16:53:51,939 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-top 2023-04-27 16:53:51,946 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-top 2023-04-27 16:53:51,952 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-top 2023-04-27 16:53:51,958 DEBUG [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-top 2023-04-27 16:53:51,958 INFO [StoreOpener-0c87014d6264f49d86caaf77094e33e7-1] regionserver.HStore(310): Store=0c87014d6264f49d86caaf77094e33e7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:51,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:51,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:51,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 0c87014d6264f49d86caaf77094e33e7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62149772, jitterRate=-0.07389622926712036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:51,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 0c87014d6264f49d86caaf77094e33e7: 2023-04-27 16:53:51,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7., pid=34, masterSystemTime=1682614431840 2023-04-27 16:53:51,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:51,966 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:51,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:53:51,967 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,967 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): 0c87014d6264f49d86caaf77094e33e7/cf1 is initiating minor compaction (all files) 2023-04-27 16:53:51,967 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): 0c87014d6264f49d86caaf77094e33e7/cf2 is initiating minor compaction (all files) 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c87014d6264f49d86caaf77094e33e7/cf1 in testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c87014d6264f49d86caaf77094e33e7/cf2 in testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/.tmp, totalSize=34.9 K 2023-04-27 16:53:51,967 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/.tmp, totalSize=34.9 K 2023-04-27 16:53:51,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:51,969 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614426118 2023-04-27 16:53:51,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:51,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 537cccd40f1fb32014621fe16dc91849, NAME => 'testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.', STARTKEY => '', ENDKEY => '41'} 2023-04-27 16:53:51,969 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614426368 2023-04-27 16:53:51,969 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614426118 2023-04-27 16:53:51,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:51,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0c87014d6264f49d86caaf77094e33e7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:51,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:51,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:51,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614431969"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614431969"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614431969"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614431969"}]},"ts":"1682614431969"} 2023-04-27 16:53:51,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:51,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:51,970 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614426703 2023-04-27 16:53:51,971 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614426368 2023-04-27 16:53:51,971 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614426703 2023-04-27 16:53:51,971 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614426967 2023-04-27 16:53:51,972 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614426967 2023-04-27 16:53:51,972 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614427273 2023-04-27 16:53:51,972 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:51,974 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614427577 2023-04-27 16:53:51,974 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614427273 2023-04-27 16:53:51,975 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1 2023-04-27 16:53:51,975 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1 2023-04-27 16:53:51,975 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614427830 2023-04-27 16:53:51,975 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614427577 2023-04-27 16:53:51,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-04-27 16:53:51,976 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 537cccd40f1fb32014621fe16dc91849 columnFamilyName cf1 2023-04-27 16:53:51,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 0c87014d6264f49d86caaf77094e33e7, server=jenkins-hbase12.apache.org,46011,1682614416374 in 279 msec 2023-04-27 16:53:51,976 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614427830 2023-04-27 16:53:51,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, ASSIGN in 448 msec 2023-04-27 16:53:51,991 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c87014d6264f49d86caaf77094e33e7#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:51,993 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c87014d6264f49d86caaf77094e33e7#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:52,013 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-bottom 2023-04-27 16:53:52,018 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-bottom 2023-04-27 16:53:52,023 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-bottom 2023-04-27 16:53:52,032 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-bottom 2023-04-27 16:53:52,042 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-bottom 2023-04-27 16:53:52,043 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/.tmp/cf2/889fc971b03043d386d2c73cd7218e14 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/889fc971b03043d386d2c73cd7218e14 2023-04-27 16:53:52,045 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/.tmp/cf1/cbd81ee4b6154146bb9ca11b2d9604e5 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/cbd81ee4b6154146bb9ca11b2d9604e5 2023-04-27 16:53:52,050 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-bottom 2023-04-27 16:53:52,053 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c87014d6264f49d86caaf77094e33e7/cf2 of 0c87014d6264f49d86caaf77094e33e7 into 889fc971b03043d386d2c73cd7218e14(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:52,053 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c87014d6264f49d86caaf77094e33e7: 2023-04-27 16:53:52,053 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7., storeName=0c87014d6264f49d86caaf77094e33e7/cf2, priority=9, startTime=1682614431966; duration=0sec 2023-04-27 16:53:52,053 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:52,056 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-bottom 2023-04-27 16:53:52,056 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c87014d6264f49d86caaf77094e33e7/cf1 of 0c87014d6264f49d86caaf77094e33e7 into cbd81ee4b6154146bb9ca11b2d9604e5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:52,056 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(310): Store=537cccd40f1fb32014621fe16dc91849/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:52,056 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c87014d6264f49d86caaf77094e33e7: 2023-04-27 16:53:52,056 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7., storeName=0c87014d6264f49d86caaf77094e33e7/cf1, priority=9, startTime=1682614431966; duration=0sec 2023-04-27 16:53:52,056 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,056 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:52,057 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2 2023-04-27 16:53:52,057 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2 2023-04-27 16:53:52,058 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 537cccd40f1fb32014621fe16dc91849 columnFamilyName cf2 2023-04-27 16:53:52,078 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-bottom 2023-04-27 16:53:52,083 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-bottom 2023-04-27 16:53:52,088 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-bottom 2023-04-27 16:53:52,093 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-bottom 2023-04-27 16:53:52,099 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-bottom 2023-04-27 16:53:52,104 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-bottom 2023-04-27 16:53:52,108 DEBUG [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-bottom 2023-04-27 16:53:52,109 INFO [StoreOpener-537cccd40f1fb32014621fe16dc91849-1] regionserver.HStore(310): Store=537cccd40f1fb32014621fe16dc91849/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:52,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:52,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 537cccd40f1fb32014621fe16dc91849; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69916223, jitterRate=0.04183290898799896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:52,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 537cccd40f1fb32014621fe16dc91849: 2023-04-27 16:53:52,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849., pid=33, masterSystemTime=1682614431840 2023-04-27 16:53:52,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:52,117 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:52,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:53:52,117 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:53:52,117 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,117 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,117 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): 537cccd40f1fb32014621fe16dc91849/cf1 is initiating minor compaction (all files) 2023-04-27 16:53:52,117 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): 537cccd40f1fb32014621fe16dc91849/cf2 is initiating minor compaction (all files) 2023-04-27 16:53:52,117 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 537cccd40f1fb32014621fe16dc91849/cf1 in testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,117 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 537cccd40f1fb32014621fe16dc91849/cf2 in testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,118 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/.tmp, totalSize=34.9 K 2023-04-27 16:53:52,118 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/.tmp, totalSize=34.9 K 2023-04-27 16:53:52,119 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614426118 2023-04-27 16:53:52,119 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614426118 2023-04-27 16:53:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,119 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614426368 2023-04-27 16:53:52,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,119 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614426368 2023-04-27 16:53:52,120 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614426703 2023-04-27 16:53:52,120 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614426703 2023-04-27 16:53:52,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=537cccd40f1fb32014621fe16dc91849, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:52,120 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614432120"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614432120"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614432120"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614432120"}]},"ts":"1682614432120"} 2023-04-27 16:53:52,120 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614426967 2023-04-27 16:53:52,120 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614426967 2023-04-27 16:53:52,121 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614427273 2023-04-27 16:53:52,121 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614427273 2023-04-27 16:53:52,121 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614427577 2023-04-27 16:53:52,121 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614427577 2023-04-27 16:53:52,122 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614427830 2023-04-27 16:53:52,122 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614427830 2023-04-27 16:53:52,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-04-27 16:53:52,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 537cccd40f1fb32014621fe16dc91849, server=jenkins-hbase12.apache.org,46011,1682614416374 in 435 msec 2023-04-27 16:53:52,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-04-27 16:53:52,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, ASSIGN in 597 msec 2023-04-27 16:53:52,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 in 4.0310 sec 2023-04-27 16:53:52,135 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 537cccd40f1fb32014621fe16dc91849#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:52,135 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 537cccd40f1fb32014621fe16dc91849#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:53:52,161 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/.tmp/cf1/caf92a34e4374caa9183c76a28dcc90b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/caf92a34e4374caa9183c76a28dcc90b 2023-04-27 16:53:52,166 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/.tmp/cf2/0d2bec61e85b48bc84d57fdb2e30f7a6 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0d2bec61e85b48bc84d57fdb2e30f7a6 2023-04-27 16:53:52,166 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 537cccd40f1fb32014621fe16dc91849/cf1 of 537cccd40f1fb32014621fe16dc91849 into caf92a34e4374caa9183c76a28dcc90b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:52,166 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 537cccd40f1fb32014621fe16dc91849: 2023-04-27 16:53:52,167 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849., storeName=537cccd40f1fb32014621fe16dc91849/cf1, priority=9, startTime=1682614432116; duration=0sec 2023-04-27 16:53:52,167 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:52,172 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 537cccd40f1fb32014621fe16dc91849/cf2 of 537cccd40f1fb32014621fe16dc91849 into 0d2bec61e85b48bc84d57fdb2e30f7a6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:53:52,172 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 537cccd40f1fb32014621fe16dc91849: 2023-04-27 16:53:52,172 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849., storeName=537cccd40f1fb32014621fe16dc91849/cf2, priority=9, startTime=1682614432117; duration=0sec 2023-04-27 16:53:52,172 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:53:52,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46011] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:44546 deadline: 1682614492270, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc. is not online on jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:52,383 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:53:52,393 INFO [Listener at localhost.localdomain/45365] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-27 16:53:52,396 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:53:52,425 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:52,427 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-04-27 16:53:52,427 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitWithoutPONR 2023-04-27 16:53:52,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:52,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-27 16:53:52,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614432432"}]},"ts":"1682614432432"} 2023-04-27 16:53:52,433 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-04-27 16:53:52,505 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-04-27 16:53:52,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, UNASSIGN}] 2023-04-27 16:53:52,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, UNASSIGN 2023-04-27 16:53:52,508 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0c87014d6264f49d86caaf77094e33e7, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:52,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614432507"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614432507"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614432507"}]},"ts":"1682614432507"} 2023-04-27 16:53:52,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 0c87014d6264f49d86caaf77094e33e7, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:52,512 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, UNASSIGN 2023-04-27 16:53:52,513 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=537cccd40f1fb32014621fe16dc91849, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:52,513 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614432513"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614432513"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614432513"}]},"ts":"1682614432513"} 2023-04-27 16:53:52,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 537cccd40f1fb32014621fe16dc91849, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:53:52,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-27 16:53:52,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:52,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 0c87014d6264f49d86caaf77094e33e7, disabling compactions & flushes 2023-04-27 16:53:52,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:52,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:52,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. after waiting 0 ms 2023-04-27 16:53:52,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:52,675 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-top] to archive 2023-04-27 16:53:52,676 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:52,678 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,679 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,681 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,682 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,683 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,685 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,686 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,688 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-top] to archive 2023-04-27 16:53:52,689 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:52,691 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,692 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,693 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,694 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,696 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,697 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,698 DEBUG [StoreCloser-testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:53:52,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:52,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7. 2023-04-27 16:53:52,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 0c87014d6264f49d86caaf77094e33e7: 2023-04-27 16:53:52,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:52,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 537cccd40f1fb32014621fe16dc91849, disabling compactions & flushes 2023-04-27 16:53:52,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. after waiting 0 ms 2023-04-27 16:53:52,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,705 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0c87014d6264f49d86caaf77094e33e7, regionState=CLOSED 2023-04-27 16:53:52,705 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614432705"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614432705"}]},"ts":"1682614432705"} 2023-04-27 16:53:52,706 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994-bottom] to archive 2023-04-27 16:53:52,707 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:52,709 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-04-27 16:53:52,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 0c87014d6264f49d86caaf77094e33e7, server=jenkins-hbase12.apache.org,46011,1682614416374 in 198 msec 2023-04-27 16:53:52,711 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c87014d6264f49d86caaf77094e33e7, UNASSIGN in 203 msec 2023-04-27 16:53:52,712 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,713 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,715 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,716 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,718 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,720 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60-bottom] to archive 2023-04-27 16:53:52,721 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:53:52,722 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,723 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,725 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,726 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,727 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,729 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,730 DEBUG [StoreCloser-testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:52,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:53:52,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-27 16:53:52,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:52,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849. 2023-04-27 16:53:52,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 537cccd40f1fb32014621fe16dc91849: 2023-04-27 16:53:52,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:52,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=537cccd40f1fb32014621fe16dc91849, regionState=CLOSED 2023-04-27 16:53:52,738 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614432737"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614432737"}]},"ts":"1682614432737"} 2023-04-27 16:53:52,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-04-27 16:53:52,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 537cccd40f1fb32014621fe16dc91849, server=jenkins-hbase12.apache.org,46011,1682614416374 in 225 msec 2023-04-27 16:53:52,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-04-27 16:53:52,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=537cccd40f1fb32014621fe16dc91849, UNASSIGN in 235 msec 2023-04-27 16:53:52,743 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614432743"}]},"ts":"1682614432743"} 2023-04-27 16:53:52,745 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-04-27 16:53:52,767 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-04-27 16:53:52,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 341 msec 2023-04-27 16:53:53,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-27 16:53:53,036 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-04-27 16:53:53,037 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitWithoutPONR 2023-04-27 16:53:53,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,040 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-04-27 16:53:53,041 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,047 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,047 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:53,047 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:53,049 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/recovered.edits] 2023-04-27 16:53:53,050 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/recovered.edits] 2023-04-27 16:53:53,050 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/recovered.edits] 2023-04-27 16:53:53,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/.regioninfo to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/.regioninfo 2023-04-27 16:53:53,053 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/.regioninfo to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/.regioninfo 2023-04-27 16:53:53,053 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/06813a281dbb4c718f9306c2cfbd29a6 2023-04-27 16:53:53,054 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/0c7ceab6e9a5438ca482a90c20f3a405 2023-04-27 16:53:53,055 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,055 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/5c18ea1cdd004dcaba690b45d41feda2 2023-04-27 16:53:53,055 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/06813a281dbb4c718f9306c2cfbd29a6.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,056 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,057 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/692281cca0fd407a97afd79588c9477d 2023-04-27 16:53:53,057 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/0c7ceab6e9a5438ca482a90c20f3a405.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,058 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,058 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d 2023-04-27 16:53:53,058 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/5c18ea1cdd004dcaba690b45d41feda2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,059 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,059 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/692281cca0fd407a97afd79588c9477d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,059 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/97d490c5459244b0873555e792b72994 2023-04-27 16:53:53,065 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,066 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf1/fc4c509131e749d3b32efb5c6d9085b2 2023-04-27 16:53:53,066 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/8ae0538d45b54d43bd2ba3f5761a0f5d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,067 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,067 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/97d490c5459244b0873555e792b72994.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,068 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,069 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/0172ac632b42489390ae05ba57baca60 2023-04-27 16:53:53,069 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf1/fc4c509131e749d3b32efb5c6d9085b2.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,071 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/1e05060b346b4d2e8b32b884d487a0ee 2023-04-27 16:53:53,072 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,072 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7 2023-04-27 16:53:53,073 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/0172ac632b42489390ae05ba57baca60.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,074 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,074 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/4a585a41e54f4be69a1d9724e2e795bf 2023-04-27 16:53:53,075 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,075 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d 2023-04-27 16:53:53,076 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/1e05060b346b4d2e8b32b884d487a0ee.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,077 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/a72b6106ab344b2ab73aaee2d427fb8f 2023-04-27 16:53:53,078 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,078 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/3f616dd9d8fe46589fdf7ab1e5ee53a7.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,080 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/cf2/cb6b28687db04cc8b0f29dc0eacd9740 2023-04-27 16:53:53,080 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,081 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/4a585a41e54f4be69a1d9724e2e795bf.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,082 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,082 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/61c26ff5c11642dbbe3b2a2bfd53dd3d.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,083 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/0c87014d6264f49d86caaf77094e33e7/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,083 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/a72b6106ab344b2ab73aaee2d427fb8f.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,083 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc/recovered.edits/85.seqid 2023-04-27 16:53:53,084 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,085 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/537cccd40f1fb32014621fe16dc91849/cf2/cb6b28687db04cc8b0f29dc0eacd9740.985ef773fc4ca76ee5d569ef4a9b03cc 2023-04-27 16:53:53,086 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/cbd81ee4b6154146bb9ca11b2d9604e5 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf1/cbd81ee4b6154146bb9ca11b2d9604e5 2023-04-27 16:53:53,087 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/caf92a34e4374caa9183c76a28dcc90b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf1/caf92a34e4374caa9183c76a28dcc90b 2023-04-27 16:53:53,088 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/889fc971b03043d386d2c73cd7218e14 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/cf2/889fc971b03043d386d2c73cd7218e14 2023-04-27 16:53:53,089 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0d2bec61e85b48bc84d57fdb2e30f7a6 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/cf2/0d2bec61e85b48bc84d57fdb2e30f7a6 2023-04-27 16:53:53,091 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7/recovered.edits/90.seqid 2023-04-27 16:53:53,092 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:53,093 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849/recovered.edits/90.seqid 2023-04-27 16:53:53,093 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitWithoutPONR/537cccd40f1fb32014621fe16dc91849 2023-04-27 16:53:53,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-04-27 16:53:53,096 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,099 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-04-27 16:53:53,101 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-04-27 16:53:53,102 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,102 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-04-27 16:53:53,102 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614433102"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:53,103 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614433102"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:53,103 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614433102"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:53,105 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:53:53,105 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 985ef773fc4ca76ee5d569ef4a9b03cc, NAME => 'testSplitWithoutPONR,,1682614425612.985ef773fc4ca76ee5d569ef4a9b03cc.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 537cccd40f1fb32014621fe16dc91849, NAME => 'testSplitWithoutPONR,,1682614428096.537cccd40f1fb32014621fe16dc91849.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0c87014d6264f49d86caaf77094e33e7, NAME => 'testSplitWithoutPONR,41,1682614428096.0c87014d6264f49d86caaf77094e33e7.', STARTKEY => '41', ENDKEY => ''}] 2023-04-27 16:53:53,105 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-04-27 16:53:53,105 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614433105"}]},"ts":"9223372036854775807"} 2023-04-27 16:53:53,107 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-04-27 16:53:53,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-04-27 16:53:53,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:53,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 143 msec 2023-04-27 16:53:53,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-04-27 16:53:53,344 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-04-27 16:53:53,357 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395 (was 390) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60976 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37944 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:34040 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37892 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45450 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:34026 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:34058 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39482 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45464 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45428 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:38002 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:37858 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60376 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45458 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45444 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:45392 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=846 (was 820) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=425 (was 411) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=11798 (was 11931) 2023-04-27 16:53:53,374 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395, OpenFileDescriptor=846, MaxFileDescriptor=60000, SystemLoadAverage=425, ProcessCount=174, AvailableMemoryMB=11797 2023-04-27 16:53:53,374 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:53:53,378 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:53:53,379 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:53:53,380 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:53,382 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:53:53,383 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:53:53,385 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,386 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c empty. 2023-04-27 16:53:53,386 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,386 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-04-27 16:53:53,397 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-04-27 16:53:53,398 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 816f71b9fefe9bb30d3dc5b1086f6f2c, NAME => 'testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 816f71b9fefe9bb30d3dc5b1086f6f2c, disabling compactions & flushes 2023-04-27 16:53:53,406 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. after waiting 0 ms 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,406 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,406 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:53,409 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:53:53,410 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614433409"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614433409"}]},"ts":"1682614433409"} 2023-04-27 16:53:53,411 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:53:53,413 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:53:53,413 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614433413"}]},"ts":"1682614433413"} 2023-04-27 16:53:53,415 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-04-27 16:53:53,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:53:53,464 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:53:53,464 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:53:53,464 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:53:53,464 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:53:53,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN}] 2023-04-27 16:53:53,466 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:53:53,466 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:53:53,617 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:53:53,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614433618"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614433618"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614433618"}]},"ts":"1682614433618"} 2023-04-27 16:53:53,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:53:53,772 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,773 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-27 16:53:53,776 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:49976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-27 16:53:53,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 816f71b9fefe9bb30d3dc5b1086f6f2c, NAME => 'testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:53:53,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:53:53,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:53:53,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,785 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,787 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 2023-04-27 16:53:53,787 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 2023-04-27 16:53:53,788 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 816f71b9fefe9bb30d3dc5b1086f6f2c columnFamilyName cf1 2023-04-27 16:53:53,788 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(310): Store=816f71b9fefe9bb30d3dc5b1086f6f2c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:53,788 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,790 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 2023-04-27 16:53:53,790 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 2023-04-27 16:53:53,790 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 816f71b9fefe9bb30d3dc5b1086f6f2c columnFamilyName cf2 2023-04-27 16:53:53,791 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(310): Store=816f71b9fefe9bb30d3dc5b1086f6f2c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:53:53,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:53:53,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:53,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:53:53,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 816f71b9fefe9bb30d3dc5b1086f6f2c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63385704, jitterRate=-0.05547940731048584}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:53:53,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:53,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c., pid=43, masterSystemTime=1682614433772 2023-04-27 16:53:53,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614433803"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614433803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614433803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614433803"}]},"ts":"1682614433803"} 2023-04-27 16:53:53,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-04-27 16:53:53,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 in 185 msec 2023-04-27 16:53:53,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-04-27 16:53:53,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN in 343 msec 2023-04-27 16:53:53,811 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:53:53,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614433811"}]},"ts":"1682614433811"} 2023-04-27 16:53:53,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-04-27 16:53:53,832 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:53:53,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 453 msec 2023-04-27 16:53:53,883 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-27 16:53:53,887 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:49980, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-27 16:53:53,894 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:53,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:53,896 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:53,896 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:53,896 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:53,896 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:53,897 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:53,897 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,905 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,905 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:53,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,905 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:53,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:53,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:53,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:53,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:53,907 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:53,908 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,908 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:53,908 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:53:53,915 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,920 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 2023-04-27 16:53:53,926 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,926 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:53,926 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,926 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,926 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:53,936 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/6e0456db489944e39e99bf0d081e7ce9 2023-04-27 16:53:53,941 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 2023-04-27 16:53:53,946 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:53,947 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/6e0456db489944e39e99bf0d081e7ce9 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 2023-04-27 16:53:53,952 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:53:53,953 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 45ms, sequenceid=6, compaction requested=false 2023-04-27 16:53:53,953 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-04-27 16:53:53,954 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:53,954 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:53,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:53,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:53,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:53,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:53,967 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,967 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:53,967 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,967 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:53,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:53,969 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:53,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14fcb8b4[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:53,969 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,978 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,978 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,978 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:53,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:53,979 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:53,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:53,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:53,979 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:53,989 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:53,989 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:53,989 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:53,999 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:53,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:53,999 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,000 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,000 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,000 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,000 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:54,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:54,003 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:54,003 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:54,003 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:54,003 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:54,003 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,011 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,011 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,011 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,013 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,013 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,013 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,021 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,021 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,021 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,021 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:54,022 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:54,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,122 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:54,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:54,141 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:54,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:54,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:54,143 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,143 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,143 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:54,143 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:54,143 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,144 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,147 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,147 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,147 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,149 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:54,150 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,150 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:54,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,150 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:54,157 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,164 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/720eb30d02464957878c32215d2fc8e9 2023-04-27 16:53:54,168 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,168 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,168 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,168 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,168 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,178 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/89fc056f0f7f40de9159edf13bcd1576 2023-04-27 16:53:54,183 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/720eb30d02464957878c32215d2fc8e9 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 2023-04-27 16:53:54,188 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:54,189 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/89fc056f0f7f40de9159edf13bcd1576 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 2023-04-27 16:53:54,194 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:53:54,195 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 45ms, sequenceid=19, compaction requested=false 2023-04-27 16:53:54,195 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:54,196 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,196 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:54,196 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:54,196 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,196 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,196 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,209 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,209 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,210 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,210 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:54,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7609cb5d[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:54,211 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:54,212 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,221 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,263 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,263 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,284 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,284 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,284 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:54,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:54,287 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:54,288 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:54,288 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:54,288 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:54,288 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,296 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,297 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,297 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,298 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,306 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:54,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,305 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,306 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,306 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:54,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,306 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,406 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:54,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:54,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:54,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:54,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:54,426 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,427 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,427 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:54,427 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:54,427 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,427 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,432 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,432 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,433 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,433 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:54,434 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:54,434 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,434 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:54,434 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:54,441 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,447 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/a756851705484605ab929383e29b443c 2023-04-27 16:53:54,452 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,452 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,456 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,457 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,457 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,467 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/21c3d2e216a248e5af85653318b2b940 2023-04-27 16:53:54,471 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/a756851705484605ab929383e29b443c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c 2023-04-27 16:53:54,475 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:54,476 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/21c3d2e216a248e5af85653318b2b940 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 2023-04-27 16:53:54,480 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:53:54,481 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 47ms, sequenceid=32, compaction requested=true 2023-04-27 16:53:54,481 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:54,481 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,481 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:54,481 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:54,481 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,481 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,481 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,494 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,494 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,495 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,495 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:54,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13ac6f81[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:54,497 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:54,497 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,505 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:54,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,505 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,505 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,505 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,506 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,506 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:54,505 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,506 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:54,506 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:54,515 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,515 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,515 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,526 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:54,529 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:54,529 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:54,529 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:54,529 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:54,529 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,536 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,537 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,538 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,538 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,539 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,539 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,540 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,580 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,580 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,581 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:54,580 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:54,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,581 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,681 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:54,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:54,699 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:54,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:54,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:54,701 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,701 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,701 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:54,701 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:54,701 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,701 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,715 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,716 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,717 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,717 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,717 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,718 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:54,726 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,727 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/2d9b45a06ead489490e7ed3e0f3b75ed 2023-04-27 16:53:54,736 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,736 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,736 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,737 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,737 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,743 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/7580486cb13448f0be2dee3b69bfb549 2023-04-27 16:53:54,748 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/2d9b45a06ead489490e7ed3e0f3b75ed as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed 2023-04-27 16:53:54,752 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:54,754 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/7580486cb13448f0be2dee3b69bfb549 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 2023-04-27 16:53:54,759 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:53:54,760 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 43ms, sequenceid=45, compaction requested=true 2023-04-27 16:53:54,760 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:54,760 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,760 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:54,760 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:54,760 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,760 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,761 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,768 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,768 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,768 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,768 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:54,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c195d10[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:54,770 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:54,770 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,778 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,778 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,778 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,779 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:54,779 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:54,789 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,789 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,789 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,799 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,799 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,799 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,800 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,800 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,800 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,800 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:54,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:54,803 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:54,803 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:54,803 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:54,803 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:54,803 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:54,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,811 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:54,811 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:54,811 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,812 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:54,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:54,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,821 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:54,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:54,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,821 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:54,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,893 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:53:54,922 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:54,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:54,942 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:54,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:54,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:54,943 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:54,944 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,944 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:54,944 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:54,944 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,944 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,956 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,956 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,956 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:54,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,956 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:54,957 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:54,957 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,958 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,960 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:54,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:54,960 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:54,960 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:54,960 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:54,960 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:54,968 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,973 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/39147323e76c48d6b000ef80c667db15 2023-04-27 16:53:54,978 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,978 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,978 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,979 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:54,979 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:54,986 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/ccf75b74168441529e3bb9fa6636ca09 2023-04-27 16:53:54,991 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/39147323e76c48d6b000ef80c667db15 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 2023-04-27 16:53:54,995 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:54,996 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/ccf75b74168441529e3bb9fa6636ca09 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 2023-04-27 16:53:55,001 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:53:55,002 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 42ms, sequenceid=58, compaction requested=true 2023-04-27 16:53:55,002 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:55,002 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:55,002 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:55,002 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:55,002 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,002 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:55,002 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:55,010 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,010 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,010 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,011 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:55,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d51653f[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:55,012 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:55,012 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,021 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,021 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:55,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,021 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:55,022 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:55,061 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,061 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,061 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,083 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,083 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:55,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:55,086 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:55,086 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:55,086 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:55,086 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:55,086 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,094 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,095 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,096 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,097 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,098 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,098 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,105 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:55,105 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,105 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,105 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,106 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,206 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:55,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:55,225 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:53:55,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:53:55,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-27 16:53:55,227 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-27 16:53:55,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:55,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-27 16:53:55,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:53:55,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,231 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,231 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,231 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,231 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,231 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:53:55,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:55,232 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:55,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:55,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-27 16:53:55,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:55,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:53:55,233 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:55,233 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. started... 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:55,233 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:55,234 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:53:55,241 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,242 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/b4be1bca098e4762a249485f7c35a4f1 2023-04-27 16:53:55,252 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,252 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,252 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:55,252 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,252 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:55,254 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 2023-04-27 16:53:55,260 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/b4be1bca098e4762a249485f7c35a4f1 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 2023-04-27 16:53:55,264 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:55,265 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 2023-04-27 16:53:55,270 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:53:55,271 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 38ms, sequenceid=71, compaction requested=true 2023-04-27 16:53:55,271 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:55,271 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:55,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:53:55,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:53:55,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-27 16:53:55,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-27 16:53:55,284 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,284 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,284 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,284 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:53:55,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-27 16:53:55,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ba02eb[Count = 0] remaining members to acquire global barrier 2023-04-27 16:53:55,286 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:53:55,286 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,295 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:53:55,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-27 16:53:55,295 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-27 16:53:55,305 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,305 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-27 16:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:53:55,308 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-27 16:53:55,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:53:55,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:53:55,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-27 16:53:55,308 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,315 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:53:55,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,316 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,316 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:53:55,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,316 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,317 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,318 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,318 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:53:55,319 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-27 16:53:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,357 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-27 16:53:55,357 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:53:55,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,357 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:53:55,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:53:55,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:53:55,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-27 16:53:55,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:53:55,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:53:55,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:53:55,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-27 16:53:55,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-27 16:53:55,458 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:53:55,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:53:55,721 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-27 16:53:55,722 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:55,724 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,726 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:55,726 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:55,726 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:55,974 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,974 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:55,974 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:55,974 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:55,974 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-04-27 16:53:55,975 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:55,975 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:55,975 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:55,975 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:55,976 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:55,977 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-27 16:53:55,977 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:55,978 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:55,978 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:55,978 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:55,979 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:55,979 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:55,979 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:55,979 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:55,980 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:55,980 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:55,980 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:55,980 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:55,981 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:55,981 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,981 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:55,982 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,982 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,982 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,982 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-27 16:53:55,983 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:55,984 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:55,984 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614465984 2023-04-27 16:53:55,985 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:55,985 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,987 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:55,988 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:55,988 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:55,988 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614495988 2023-04-27 16:53:55,988 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614555988 2023-04-27 16:53:55,988 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:53:55,988 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:55,991 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:55,991 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:55,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:55,991 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:56,239 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:56,239 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:56,240 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:56,241 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-27 16:53:56,241 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:56,242 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:56,242 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:56,242 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:56,242 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,243 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:56,243 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,243 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,243 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,244 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:56,244 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,245 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,245 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,245 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:56,246 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:56,246 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:56,247 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614466247 2023-04-27 16:53:56,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,248 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:56,251 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:56,251 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:56,252 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:56,252 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614496252 2023-04-27 16:53:56,252 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614556252 2023-04-27 16:53:56,252 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:56,252 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:56,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN}] 2023-04-27 16:53:56,255 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:56,255 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:56,255 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:56,502 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,502 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:56,502 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:56,503 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:56,503 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:56,503 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:56,503 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:56,503 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:56,503 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:56,504 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:56,505 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-27 16:53:56,505 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:56,506 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:56,506 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:56,506 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:56,506 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,507 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:56,507 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,507 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,507 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,508 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:56,508 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,508 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:56,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,509 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:56,509 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,509 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,509 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,510 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:56,510 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:56,511 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:56,511 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614466511 2023-04-27 16:53:56,512 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:56,513 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,515 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:56,516 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:56,516 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:56,516 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614496516 2023-04-27 16:53:56,516 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614556516 2023-04-27 16:53:56,516 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:56,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN}] 2023-04-27 16:53:56,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:56,519 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:56,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:56,520 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614436520"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614436520"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614436520"}]},"ts":"1682614436520"} 2023-04-27 16:53:56,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:53:56,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:56,522 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:56,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:56,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:56,766 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:56,767 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:56,767 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:56,768 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:56,769 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:56,769 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:53:56,770 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:56,770 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:56,770 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:56,770 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:56,771 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,771 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:56,771 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,771 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:56,771 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,772 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:56,772 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,772 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:56,772 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:56,773 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,773 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:56,773 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:56,773 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,773 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,773 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:56,773 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:56,773 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:56,774 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:56,774 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:56,775 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:56,775 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:53:56,775 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614466775 2023-04-27 16:53:56,776 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:56,776 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:56,778 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:56,778 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:56,778 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:56,778 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614496778 2023-04-27 16:53:56,778 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614556778 2023-04-27 16:53:56,779 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:53:56,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:56,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614436779"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614436779"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614436779"}]},"ts":"1682614436779"} 2023-04-27 16:53:56,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:53:56,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:56,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:56,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 816f71b9fefe9bb30d3dc5b1086f6f2c, disabling compactions & flushes 2023-04-27 16:53:56,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:56,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:56,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. after waiting 0 ms 2023-04-27 16:53:56,935 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:56,935 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 816f71b9fefe9bb30d3dc5b1086f6f2c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:53:56,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 2023-04-27 16:53:56,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/d3f4cf0be09d49dc8420c874414d5a88 2023-04-27 16:53:56,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 2023-04-27 16:53:56,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:56,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/d3f4cf0be09d49dc8420c874414d5a88 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 2023-04-27 16:53:56,975 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:53:56,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 816f71b9fefe9bb30d3dc5b1086f6f2c in 41ms, sequenceid=82, compaction requested=true 2023-04-27 16:53:56,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:53:56,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:53:56,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:53:56,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:53:56,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:56,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED 2023-04-27 16:53:56,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614436988"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614436988"}]},"ts":"1682614436988"} 2023-04-27 16:53:56,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:56,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:56,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:56,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:57,029 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,029 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:57,029 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:57,029 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:57,030 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:57,030 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:57,030 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:57,030 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:57,030 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:57,031 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:57,032 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:57,032 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-27 16:53:57,032 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-04-27 16:53:57,032 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:57,032 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:57,033 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:57,033 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:57,033 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,033 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,034 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,035 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:57,035 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,035 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,035 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:57,036 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:57,036 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,036 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,036 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,036 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:57,036 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,036 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:57,036 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:57,037 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:57,038 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:57,038 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:53:57,038 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614467038 2023-04-27 16:53:57,038 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:57,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:57,041 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:57,042 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:57,042 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:57,042 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614497042 2023-04-27 16:53:57,042 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614557042 2023-04-27 16:53:57,042 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:57,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:57,042 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614437042"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614437042"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614437042"}]},"ts":"1682614437042"} 2023-04-27 16:53:57,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:57,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-04-27 16:53:57,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 in 263 msec 2023-04-27 16:53:57,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:57,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:57,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:57,292 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,292 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:57,292 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:57,292 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:57,295 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:57,295 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:57,295 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:57,295 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:57,295 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:57,296 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:57,296 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:57,297 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-27 16:53:57,297 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:57,297 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:57,298 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:57,298 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:57,298 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:57,298 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,299 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:57,299 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,299 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,299 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,300 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:57,300 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,300 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,300 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:57,301 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,301 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,301 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:57,301 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:57,302 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,302 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,302 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,302 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:57,302 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:57,302 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-27 16:53:57,303 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:57,303 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:57,304 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:53:57,304 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614467304 2023-04-27 16:53:57,305 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:57,305 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,307 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:57,307 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:57,308 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:57,308 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614497308 2023-04-27 16:53:57,308 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614557308 2023-04-27 16:53:57,308 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:57,308 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:57,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-04-27 16:53:57,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN in 793 msec 2023-04-27 16:53:57,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:57,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:57,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:57,558 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,558 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:57,558 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:57,558 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:57,559 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:57,559 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:57,559 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:57,559 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:57,559 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:57,560 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:57,561 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:57,562 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,562 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:57,562 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,562 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:57,563 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,563 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:57,563 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,563 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:57,564 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:57,564 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,564 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:57,564 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:57,565 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:57,565 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:57,565 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:57,565 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:57,565 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:57,566 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:57,568 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614467568 2023-04-27 16:53:57,569 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,569 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:57,572 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:57,572 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:57,572 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:57,572 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614497572 2023-04-27 16:53:57,572 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614557572 2023-04-27 16:53:57,572 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:53:57,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:57,585 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=816f71b9fefe9bb30d3dc5b1086f6f2c, threads=14 2023-04-27 16:53:57,586 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,586 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,586 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,586 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,587 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,588 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,588 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,793 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,800 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,815 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,816 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,820 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,832 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,848 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,855 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,855 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,856 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,856 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,856 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,862 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,862 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:57,862 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 816f71b9fefe9bb30d3dc5b1086f6f2c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:53:57,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:57,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:57,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:57,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:58,073 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:58,073 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:58,074 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:58,075 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-27 16:53:58,075 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:58,075 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:58,076 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:58,076 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:58,077 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:58,077 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:58,077 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,077 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:58,077 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,078 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,078 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,078 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:58,078 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,079 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,079 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:58,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:58,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:58,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:58,080 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,080 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,080 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,081 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-27 16:53:58,082 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:58,082 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:58,083 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614468083 2023-04-27 16:53:58,084 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,084 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:58,087 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:58,087 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:58,087 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:58,087 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614498087 2023-04-27 16:53:58,088 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614558088 2023-04-27 16:53:58,088 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:58,097 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=816f71b9fefe9bb30d3dc5b1086f6f2c, threads=14 2023-04-27 16:53:58,098 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,100 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,105 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,105 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,108 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,108 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,109 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,109 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,112 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,113 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,116 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,120 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,120 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,123 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,355 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,356 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,375 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,388 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,392 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,405 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,428 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,436 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,438 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,439 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,439 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,439 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,441 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,441 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 for region: 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:53:58,441 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 816f71b9fefe9bb30d3dc5b1086f6f2c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:53:58,483 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:58,489 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/b916b9c9bd3e02ea62ae1ce105b3e45a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:58,491 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/c7106e59e45afb6f541ec61fd2406ddc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:53:58,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:58,491 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:58,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:58,588 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,588 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:58,588 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:58,588 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:58,589 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:58,589 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:58,589 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:58,589 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:58,589 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:58,590 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:58,591 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-27 16:53:58,591 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:58,591 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:58,591 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:58,592 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:58,592 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:58,592 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:58,592 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,592 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,593 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,594 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:58,594 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,594 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:58,594 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:58,596 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:58,596 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,597 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,597 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,597 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,597 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,597 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:53:58,598 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:58,598 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:58,599 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614468599 2023-04-27 16:53:58,601 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,601 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:58,603 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:58,603 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:58,603 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:58,603 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614498603 2023-04-27 16:53:58,603 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614558603 2023-04-27 16:53:58,603 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:53:58,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:58,611 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:58,611 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:58,611 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:58,854 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,854 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:58,854 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:58,854 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:58,855 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:58,855 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:58,856 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:58,856 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:58,856 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:58,857 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:58,857 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-27 16:53:58,857 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:58,857 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:58,858 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:58,858 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:58,858 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:58,858 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:58,858 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:58,859 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:58,860 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:58,860 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,860 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,861 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,861 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,861 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:58,861 DEBUG [Listener at localhost.localdomain/45365] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,861 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:58,862 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:58,862 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:58,862 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614468862 2023-04-27 16:53:58,865 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:58,865 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:58,867 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:58,867 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:58,867 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:58,868 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614498868 2023-04-27 16:53:58,868 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614558868 2023-04-27 16:53:58,868 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:53:58,868 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:58,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:58,879 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:53:58,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:58,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:59,119 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,119 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:59,119 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:59,119 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:59,119 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:59,120 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:59,120 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:59,120 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:59,120 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:59,121 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:59,122 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:59,123 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:59,123 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,123 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:59,123 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,123 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,124 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,124 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:59,124 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,124 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,125 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:59,125 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,126 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:59,126 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:59,126 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:59,126 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,126 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,126 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,126 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,126 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,126 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:53:59,127 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:59,127 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:59,128 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614469128 2023-04-27 16:53:59,130 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:59,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,133 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:59,133 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:59,133 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:59,133 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614499133 2023-04-27 16:53:59,134 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614559134 2023-04-27 16:53:59,134 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:59,134 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:59,137 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:59,137 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:53:59,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:59,137 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:59,385 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,385 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:59,385 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:59,385 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:59,386 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:59,386 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:59,386 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:59,386 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:59,386 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:59,387 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:59,388 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-27 16:53:59,388 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:59,388 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:59,388 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:59,388 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,389 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,390 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:59,390 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,390 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,390 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:59,391 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,391 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,391 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,391 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,391 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:59,392 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:59,393 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:59,393 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614469393 2023-04-27 16:53:59,395 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:59,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,398 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:59,398 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:59,398 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:59,398 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614499398 2023-04-27 16:53:59,398 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614559398 2023-04-27 16:53:59,398 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:53:59,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:59,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:59,402 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:53:59,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:59,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:59,648 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,648 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:59,648 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:59,648 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:59,649 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:59,650 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:59,650 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:59,650 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:59,650 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:59,650 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:59,651 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-27 16:53:59,651 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:59,651 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:59,652 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:59,652 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:59,652 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:59,652 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:59,652 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,653 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,654 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:59,654 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,654 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:59,654 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:59,654 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:59,655 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,655 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,655 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,655 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,655 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,655 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:59,656 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:59,656 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:59,656 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614469656 2023-04-27 16:53:59,658 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:59,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,660 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:59,660 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:59,660 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:59,660 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614499660 2023-04-27 16:53:59,660 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614559660 2023-04-27 16:53:59,661 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:53:59,661 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:59,665 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN] 2023-04-27 16:53:59,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:59,665 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:53:59,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:59,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:53:59,911 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:53:59,911 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:53:59,912 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:53:59,913 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-27 16:53:59,913 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:53:59,913 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:53:59,913 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:53:59,913 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:53:59,914 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,915 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:53:59,915 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,915 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:53:59,915 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:53:59,916 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,916 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,916 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,916 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,917 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:53:59,917 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:53:59,917 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:53:59,918 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:53:59,918 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:53:59,918 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614469918 2023-04-27 16:53:59,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:53:59,920 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:53:59,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:53:59,924 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:53:59,924 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:53:59,924 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:53:59,924 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614499924 2023-04-27 16:53:59,925 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614559925 2023-04-27 16:53:59,925 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:53:59,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:53:59,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:53:59,931 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:53:59,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:53:59,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:54:00,175 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,175 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:54:00,175 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:54:00,176 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:54:00,176 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:54:00,176 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:54:00,176 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:54:00,176 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:54:00,176 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:54:00,177 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:54:00,178 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:54:00,178 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:54:00,178 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:54:00,179 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,180 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,180 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,180 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:54:00,180 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,180 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,181 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:00,181 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,181 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,181 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:54:00,181 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,182 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,182 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,182 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:54:00,182 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:54:00,183 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:54:00,183 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:54:00,183 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:54:00,184 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614470184 2023-04-27 16:54:00,185 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:54:00,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,187 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,189 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:54:00,189 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:54:00,189 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:54:00,189 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614500189 2023-04-27 16:54:00,189 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614560189 2023-04-27 16:54:00,189 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:54:00,189 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:54:00,193 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:54:00,193 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:54:00,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:54:00,193 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:54:00,439 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:54:00,440 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:54:00,440 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:54:00,441 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:54:00,442 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-27 16:54:00,442 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:54:00,442 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:54:00,442 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:54:00,443 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:54:00,443 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:54:00,443 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,443 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,444 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,445 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:00,445 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,445 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,445 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:54:00,445 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,445 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,445 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,445 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:54:00,445 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:54:00,446 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:54:00,447 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:54:00,447 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:54:00,447 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614470447 2023-04-27 16:54:00,449 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:54:00,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,451 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,453 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:54:00,454 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:54:00,454 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:54:00,454 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614500454 2023-04-27 16:54:00,454 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614560454 2023-04-27 16:54:00,454 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-27 16:54:00,454 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:54:00,457 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:54:00,457 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-27 16:54:00,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:54:00,457 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:54:00,704 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,704 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:54:00,704 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:54:00,704 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:54:00,705 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:54:00,705 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:54:00,705 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:54:00,705 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:54:00,705 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:54:00,706 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:54:00,706 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-27 16:54:00,706 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-27 16:54:00,707 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:54:00,707 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:54:00,707 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:54:00,707 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:54:00,708 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,708 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:54:00,708 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,708 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,709 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,709 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:54:00,709 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,709 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,709 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:00,710 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,710 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,710 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:54:00,710 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,710 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,710 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,710 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:54:00,710 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-27 16:54:00,711 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:54:00,712 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:54:00,712 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:54:00,712 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614470712 2023-04-27 16:54:00,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,714 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:54:00,717 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,719 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:54:00,719 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:54:00,719 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:54:00,719 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614500719 2023-04-27 16:54:00,719 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614560719 2023-04-27 16:54:00,720 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-27 16:54:00,720 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:54:00,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc exec-time=4.9970 sec 2023-04-27 16:54:00,723 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,724 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=true 2023-04-27 16:54:00,874 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:54:00,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:00,876 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614440875"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614440875"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614440875"}]},"ts":"1682614440875"} 2023-04-27 16:54:00,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:00,878 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:54:00,878 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:54:00,878 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:54:00,970 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:00,970 WARN [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-04-27 16:54:00,970 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:54:00,970 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:54:00,970 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:54:00,970 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:54:00,970 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:54:00,971 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:54:00,971 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:54:00,971 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:54:00,971 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:54:00,973 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-04-27 16:54:00,973 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:54:00,973 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:54:00,973 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:54:00,973 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:54:00,974 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,975 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:54:00,975 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:00,975 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:54:00,975 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,975 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:54:00,975 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,975 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-04-27 16:54:00,976 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:54:00,977 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:54:00,977 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c to restore RIT 2023-04-27 16:54:00,977 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614470977 2023-04-27 16:54:00,978 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:54:00,978 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:54:00,980 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:54:00,981 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPENING, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:54:00,981 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:54:00,981 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614500981 2023-04-27 16:54:00,981 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614560981 2023-04-27 16:54:00,981 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:54:00,981 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:00,981 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614440981"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614440981"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614440981"}]},"ts":"1682614440981"} 2023-04-27 16:54:00,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:01,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 816f71b9fefe9bb30d3dc5b1086f6f2c, NAME => 'testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:54:01,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:01,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:01,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,141 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,142 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 2023-04-27 16:54:01,143 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 2023-04-27 16:54:01,143 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 816f71b9fefe9bb30d3dc5b1086f6f2c columnFamilyName cf1 2023-04-27 16:54:01,150 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 2023-04-27 16:54:01,154 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed 2023-04-27 16:54:01,157 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 2023-04-27 16:54:01,162 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 2023-04-27 16:54:01,166 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 2023-04-27 16:54:01,170 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c 2023-04-27 16:54:01,173 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 2023-04-27 16:54:01,174 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(310): Store=816f71b9fefe9bb30d3dc5b1086f6f2c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:01,174 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,175 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 2023-04-27 16:54:01,175 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 2023-04-27 16:54:01,175 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 816f71b9fefe9bb30d3dc5b1086f6f2c columnFamilyName cf2 2023-04-27 16:54:01,182 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 2023-04-27 16:54:01,186 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 2023-04-27 16:54:01,190 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 2023-04-27 16:54:01,193 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 2023-04-27 16:54:01,201 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 2023-04-27 16:54:01,207 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 2023-04-27 16:54:01,212 DEBUG [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 2023-04-27 16:54:01,212 INFO [StoreOpener-816f71b9fefe9bb30d3dc5b1086f6f2c-1] regionserver.HStore(310): Store=816f71b9fefe9bb30d3dc5b1086f6f2c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:01,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.splits 2023-04-27 16:54:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:01,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 816f71b9fefe9bb30d3dc5b1086f6f2c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73908906, jitterRate=0.1013285219669342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:01,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:54:01,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c., pid=48, masterSystemTime=1682614441136 2023-04-27 16:54:01,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:01,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:01,220 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:01,220 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:01,221 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-27 16:54:01,221 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-27 16:54:01,221 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): 816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 is initiating minor compaction (all files) 2023-04-27 16:54:01,221 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1912): 816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 is initiating minor compaction (all files) 2023-04-27 16:54:01,221 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 in testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,221 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 in testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,221 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp, totalSize=34.9 K 2023-04-27 16:54:01,221 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp, totalSize=34.9 K 2023-04-27 16:54:01,223 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 0ee873e8b3eb4ce6b824ff5fe0007755, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614433888 2023-04-27 16:54:01,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,223 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 6e0456db489944e39e99bf0d081e7ce9, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614433888 2023-04-27 16:54:01,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,223 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 720eb30d02464957878c32215d2fc8e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614434124 2023-04-27 16:54:01,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:01,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614441223"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614441223"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614441223"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614441223"}]},"ts":"1682614441223"} 2023-04-27 16:54:01,224 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 89fc056f0f7f40de9159edf13bcd1576, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614434124 2023-04-27 16:54:01,224 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting a756851705484605ab929383e29b443c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614434408 2023-04-27 16:54:01,224 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 21c3d2e216a248e5af85653318b2b940, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614434408 2023-04-27 16:54:01,224 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 2d9b45a06ead489490e7ed3e0f3b75ed, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614434683 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 7580486cb13448f0be2dee3b69bfb549, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614434683 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 39147323e76c48d6b000ef80c667db15, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614434924 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting ccf75b74168441529e3bb9fa6636ca09, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614434924 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting b4be1bca098e4762a249485f7c35a4f1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614435209 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 91b7ab01f59c4f3ba520df3fd4a44da8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614435209 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 749a6ef5d1ce43bf8f801cfe73da5cc0, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614435460 2023-04-27 16:54:01,225 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting d3f4cf0be09d49dc8420c874414d5a88, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614435460 2023-04-27 16:54:01,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-04-27 16:54:01,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 in 242 msec 2023-04-27 16:54:01,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN in 1.5670 sec 2023-04-27 16:54:01,232 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-04-27 16:54:01,235 DEBUG [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-04-27 16:54:01,236 INFO [Listener at localhost.localdomain/45365] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-04-27 16:54:01,239 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-04-27 16:54:01,240 INFO [Listener at localhost.localdomain/45365] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-04-27 16:54:01,240 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-04-27 16:54:01,258 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 816f71b9fefe9bb30d3dc5b1086f6f2c#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:01,265 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] throttle.PressureAwareThroughputController(145): 816f71b9fefe9bb30d3dc5b1086f6f2c#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:01,289 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf1/256252da37e74a09b8eecc447ddf5aeb as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/256252da37e74a09b8eecc447ddf5aeb 2023-04-27 16:54:01,301 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 816f71b9fefe9bb30d3dc5b1086f6f2c/cf1 of 816f71b9fefe9bb30d3dc5b1086f6f2c into 256252da37e74a09b8eecc447ddf5aeb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:01,303 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:54:01,303 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c., storeName=816f71b9fefe9bb30d3dc5b1086f6f2c/cf1, priority=9, startTime=1682614441219; duration=0sec 2023-04-27 16:54:01,304 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:01,304 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/.tmp/cf2/9eb4dbed07934f7685d33541d54c81bc as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/9eb4dbed07934f7685d33541d54c81bc 2023-04-27 16:54:01,311 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 816f71b9fefe9bb30d3dc5b1086f6f2c/cf2 of 816f71b9fefe9bb30d3dc5b1086f6f2c into 9eb4dbed07934f7685d33541d54c81bc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:01,312 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:54:01,312 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c., storeName=816f71b9fefe9bb30d3dc5b1086f6f2c/cf2, priority=9, startTime=1682614441219; duration=0sec 2023-04-27 16:54:01,312 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:01,319 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:01,320 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-04-27 16:54:01,321 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testRollbackAndDoubleExecution 2023-04-27 16:54:01,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-27 16:54:01,325 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614441325"}]},"ts":"1682614441325"} 2023-04-27 16:54:01,326 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-04-27 16:54:01,378 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-04-27 16:54:01,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN}] 2023-04-27 16:54:01,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN 2023-04-27 16:54:01,380 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:01,381 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614441380"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614441380"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614441380"}]},"ts":"1682614441380"} 2023-04-27 16:54:01,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:01,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-27 16:54:01,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 816f71b9fefe9bb30d3dc5b1086f6f2c, disabling compactions & flushes 2023-04-27 16:54:01,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. after waiting 0 ms 2023-04-27 16:54:01,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,535 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0] to archive 2023-04-27 16:54:01,536 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:01,538 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/0ee873e8b3eb4ce6b824ff5fe0007755 2023-04-27 16:54:01,539 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/720eb30d02464957878c32215d2fc8e9 2023-04-27 16:54:01,540 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/a756851705484605ab929383e29b443c 2023-04-27 16:54:01,541 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/2d9b45a06ead489490e7ed3e0f3b75ed 2023-04-27 16:54:01,542 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/39147323e76c48d6b000ef80c667db15 2023-04-27 16:54:01,544 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/b4be1bca098e4762a249485f7c35a4f1 2023-04-27 16:54:01,545 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/749a6ef5d1ce43bf8f801cfe73da5cc0 2023-04-27 16:54:01,549 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88] to archive 2023-04-27 16:54:01,550 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:01,551 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/6e0456db489944e39e99bf0d081e7ce9 2023-04-27 16:54:01,552 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/89fc056f0f7f40de9159edf13bcd1576 2023-04-27 16:54:01,554 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/21c3d2e216a248e5af85653318b2b940 2023-04-27 16:54:01,555 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/7580486cb13448f0be2dee3b69bfb549 2023-04-27 16:54:01,556 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/ccf75b74168441529e3bb9fa6636ca09 2023-04-27 16:54:01,557 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/91b7ab01f59c4f3ba520df3fd4a44da8 2023-04-27 16:54:01,558 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/d3f4cf0be09d49dc8420c874414d5a88 2023-04-27 16:54:01,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:54:01,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:01,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c. 2023-04-27 16:54:01,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 816f71b9fefe9bb30d3dc5b1086f6f2c: 2023-04-27 16:54:01,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,566 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=816f71b9fefe9bb30d3dc5b1086f6f2c, regionState=CLOSED 2023-04-27 16:54:01,566 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614441565"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614441565"}]},"ts":"1682614441565"} 2023-04-27 16:54:01,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-04-27 16:54:01,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 816f71b9fefe9bb30d3dc5b1086f6f2c, server=jenkins-hbase12.apache.org,41143,1682614416328 in 185 msec 2023-04-27 16:54:01,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-04-27 16:54:01,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, UNASSIGN in 190 msec 2023-04-27 16:54:01,570 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614441570"}]},"ts":"1682614441570"} 2023-04-27 16:54:01,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-04-27 16:54:01,578 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-04-27 16:54:01,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 258 msec 2023-04-27 16:54:01,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-27 16:54:01,628 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-04-27 16:54:01,629 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testRollbackAndDoubleExecution 2023-04-27 16:54:01,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,632 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-04-27 16:54:01,635 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,637 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits] 2023-04-27 16:54:01,639 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/256252da37e74a09b8eecc447ddf5aeb to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf1/256252da37e74a09b8eecc447ddf5aeb 2023-04-27 16:54:01,642 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/9eb4dbed07934f7685d33541d54c81bc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/cf2/9eb4dbed07934f7685d33541d54c81bc 2023-04-27 16:54:01,644 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c/recovered.edits/90.seqid 2023-04-27 16:54:01,644 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackAndDoubleExecution/816f71b9fefe9bb30d3dc5b1086f6f2c 2023-04-27 16:54:01,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-04-27 16:54:01,647 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,649 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-04-27 16:54:01,650 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-04-27 16:54:01,652 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,652 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-04-27 16:54:01,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614441652"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:01,653 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-27 16:54:01,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 816f71b9fefe9bb30d3dc5b1086f6f2c, NAME => 'testRollbackAndDoubleExecution,,1682614433379.816f71b9fefe9bb30d3dc5b1086f6f2c.', STARTKEY => '', ENDKEY => ''}] 2023-04-27 16:54:01,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-04-27 16:54:01,654 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614441653"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:01,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-04-27 16:54:01,663 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:54:01,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 34 msec 2023-04-27 16:54:01,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-04-27 16:54:01,734 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-04-27 16:54:01,747 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400 (was 395) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:60976 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43720 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:34040 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:37892 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase12:41143-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43734 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43712 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43760 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:33898 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43800 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43740 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:45428 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:38002 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:37858 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43750 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:60376 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_396120887_15 at /127.0.0.1:43072 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=840 (was 846), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=415 (was 425), ProcessCount=174 (was 174), AvailableMemoryMB=11780 (was 11797) 2023-04-27 16:54:01,760 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400, OpenFileDescriptor=840, MaxFileDescriptor=60000, SystemLoadAverage=415, ProcessCount=174, AvailableMemoryMB=11779 2023-04-27 16:54:01,760 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:01,767 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:54:01,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:54:01,769 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:01,771 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:54:01,772 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:54:01,773 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:01,774 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 empty. 2023-04-27 16:54:01,775 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:01,775 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-04-27 16:54:01,790 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-04-27 16:54:01,791 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4da958a3a9419b0e6e2a5a7fefc50972, NAME => 'testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 4da958a3a9419b0e6e2a5a7fefc50972, disabling compactions & flushes 2023-04-27 16:54:01,801 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. after waiting 0 ms 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:01,801 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:01,801 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 4da958a3a9419b0e6e2a5a7fefc50972: 2023-04-27 16:54:01,804 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:54:01,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614441805"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614441805"}]},"ts":"1682614441805"} 2023-04-27 16:54:01,807 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:54:01,808 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:54:01,808 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614441808"}]},"ts":"1682614441808"} 2023-04-27 16:54:01,809 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-04-27 16:54:01,831 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:54:01,832 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:54:01,832 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:54:01,832 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:54:01,832 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:54:01,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, ASSIGN}] 2023-04-27 16:54:01,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, ASSIGN 2023-04-27 16:54:01,835 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:54:01,985 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:54:01,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=4da958a3a9419b0e6e2a5a7fefc50972, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:01,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614441986"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614441986"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614441986"}]},"ts":"1682614441986"} 2023-04-27 16:54:01,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 4da958a3a9419b0e6e2a5a7fefc50972, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:02,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4da958a3a9419b0e6e2a5a7fefc50972, NAME => 'testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:54:02,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:02,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:02,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,146 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,147 DEBUG [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf1 2023-04-27 16:54:02,147 DEBUG [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf1 2023-04-27 16:54:02,148 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4da958a3a9419b0e6e2a5a7fefc50972 columnFamilyName cf1 2023-04-27 16:54:02,149 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] regionserver.HStore(310): Store=4da958a3a9419b0e6e2a5a7fefc50972/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,149 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,150 DEBUG [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf2 2023-04-27 16:54:02,150 DEBUG [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf2 2023-04-27 16:54:02,150 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4da958a3a9419b0e6e2a5a7fefc50972 columnFamilyName cf2 2023-04-27 16:54:02,151 INFO [StoreOpener-4da958a3a9419b0e6e2a5a7fefc50972-1] regionserver.HStore(310): Store=4da958a3a9419b0e6e2a5a7fefc50972/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:02,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:54:02,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 4da958a3a9419b0e6e2a5a7fefc50972; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70403915, jitterRate=0.049100086092948914}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:02,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 4da958a3a9419b0e6e2a5a7fefc50972: 2023-04-27 16:54:02,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972., pid=55, masterSystemTime=1682614442140 2023-04-27 16:54:02,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=4da958a3a9419b0e6e2a5a7fefc50972, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614442161"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614442161"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614442161"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614442161"}]},"ts":"1682614442161"} 2023-04-27 16:54:02,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-04-27 16:54:02,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 4da958a3a9419b0e6e2a5a7fefc50972, server=jenkins-hbase12.apache.org,41143,1682614416328 in 174 msec 2023-04-27 16:54:02,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-04-27 16:54:02,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, ASSIGN in 332 msec 2023-04-27 16:54:02,168 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:54:02,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614442168"}]},"ts":"1682614442168"} 2023-04-27 16:54:02,170 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-04-27 16:54:02,179 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:54:02,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 411 msec 2023-04-27 16:54:02,270 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,271 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, UNASSIGN}] 2023-04-27 16:54:02,276 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, UNASSIGN 2023-04-27 16:54:02,277 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=4da958a3a9419b0e6e2a5a7fefc50972, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,277 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614442277"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614442277"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614442277"}]},"ts":"1682614442277"} 2023-04-27 16:54:02,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 4da958a3a9419b0e6e2a5a7fefc50972, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:02,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 4da958a3a9419b0e6e2a5a7fefc50972, disabling compactions & flushes 2023-04-27 16:54:02,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. after waiting 0 ms 2023-04-27 16:54:02,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-04-27 16:54:02,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:02,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972. 2023-04-27 16:54:02,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 4da958a3a9419b0e6e2a5a7fefc50972: 2023-04-27 16:54:02,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:02,440 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=4da958a3a9419b0e6e2a5a7fefc50972, regionState=CLOSED 2023-04-27 16:54:02,440 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614442440"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614442440"}]},"ts":"1682614442440"} 2023-04-27 16:54:02,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-04-27 16:54:02,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 4da958a3a9419b0e6e2a5a7fefc50972, server=jenkins-hbase12.apache.org,41143,1682614416328 in 164 msec 2023-04-27 16:54:02,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-04-27 16:54:02,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4da958a3a9419b0e6e2a5a7fefc50972, UNASSIGN in 169 msec 2023-04-27 16:54:02,476 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-04-27 16:54:02,479 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-04-27 16:54:02,481 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1682614442480"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1682614442480"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1682614442480"}]},"ts":"1682614442480"} 2023-04-27 16:54:02,481 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442480"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614442480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614442480"}]},"ts":"1682614442480"} 2023-04-27 16:54:02,481 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442480"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614442480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614442480"}]},"ts":"1682614442480"} 2023-04-27 16:54:02,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, ASSIGN}] 2023-04-27 16:54:02,486 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, ASSIGN 2023-04-27 16:54:02,486 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:54:02,487 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:54:02,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, ASSIGN 2023-04-27 16:54:02,488 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:54:02,637 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:54:02,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=bd98018c89359efe591534fa1b58d354, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442638"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614442638"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614442638"}]},"ts":"1682614442638"} 2023-04-27 16:54:02,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure bd98018c89359efe591534fa1b58d354, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:02,643 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1b60f0ca3dad9cb4e6fbe55481d58af3, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,643 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442643"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614442643"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614442643"}]},"ts":"1682614442643"} 2023-04-27 16:54:02,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 1b60f0ca3dad9cb4e6fbe55481d58af3, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:02,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:02,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bd98018c89359efe591534fa1b58d354, NAME => 'testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.', STARTKEY => '', ENDKEY => '41'} 2023-04-27 16:54:02,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:02,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:02,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,797 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,799 DEBUG [StoreOpener-bd98018c89359efe591534fa1b58d354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf1 2023-04-27 16:54:02,799 DEBUG [StoreOpener-bd98018c89359efe591534fa1b58d354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf1 2023-04-27 16:54:02,799 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bd98018c89359efe591534fa1b58d354 columnFamilyName cf1 2023-04-27 16:54:02,800 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] regionserver.HStore(310): Store=bd98018c89359efe591534fa1b58d354/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,800 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,801 DEBUG [StoreOpener-bd98018c89359efe591534fa1b58d354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf2 2023-04-27 16:54:02,801 DEBUG [StoreOpener-bd98018c89359efe591534fa1b58d354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf2 2023-04-27 16:54:02,802 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bd98018c89359efe591534fa1b58d354 columnFamilyName cf2 2023-04-27 16:54:02,802 INFO [StoreOpener-bd98018c89359efe591534fa1b58d354-1] regionserver.HStore(310): Store=bd98018c89359efe591534fa1b58d354/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:02,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:02,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened bd98018c89359efe591534fa1b58d354; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66657177, jitterRate=-0.006730660796165466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:02,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for bd98018c89359efe591534fa1b58d354: 2023-04-27 16:54:02,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354., pid=61, masterSystemTime=1682614442792 2023-04-27 16:54:02,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:02,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:02,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:02,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b60f0ca3dad9cb4e6fbe55481d58af3, NAME => 'testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.', STARTKEY => '41', ENDKEY => ''} 2023-04-27 16:54:02,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=bd98018c89359efe591534fa1b58d354, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442810"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614442810"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614442810"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614442810"}]},"ts":"1682614442810"} 2023-04-27 16:54:02,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:02,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:02,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,812 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-04-27 16:54:02,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure bd98018c89359efe591534fa1b58d354, server=jenkins-hbase12.apache.org,41143,1682614416328 in 171 msec 2023-04-27 16:54:02,814 DEBUG [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf1 2023-04-27 16:54:02,814 DEBUG [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf1 2023-04-27 16:54:02,814 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b60f0ca3dad9cb4e6fbe55481d58af3 columnFamilyName cf1 2023-04-27 16:54:02,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, ASSIGN in 328 msec 2023-04-27 16:54:02,815 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] regionserver.HStore(310): Store=1b60f0ca3dad9cb4e6fbe55481d58af3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,815 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,816 DEBUG [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf2 2023-04-27 16:54:02,816 DEBUG [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf2 2023-04-27 16:54:02,816 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b60f0ca3dad9cb4e6fbe55481d58af3 columnFamilyName cf2 2023-04-27 16:54:02,817 INFO [StoreOpener-1b60f0ca3dad9cb4e6fbe55481d58af3-1] regionserver.HStore(310): Store=1b60f0ca3dad9cb4e6fbe55481d58af3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:02,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:02,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:02,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 1b60f0ca3dad9cb4e6fbe55481d58af3; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68153235, jitterRate=0.01556234061717987}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:02,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 1b60f0ca3dad9cb4e6fbe55481d58af3: 2023-04-27 16:54:02,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3., pid=62, masterSystemTime=1682614442792 2023-04-27 16:54:02,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:02,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:02,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1b60f0ca3dad9cb4e6fbe55481d58af3, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:02,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614442825"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614442825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614442825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614442825"}]},"ts":"1682614442825"} 2023-04-27 16:54:02,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-04-27 16:54:02,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 1b60f0ca3dad9cb4e6fbe55481d58af3, server=jenkins-hbase12.apache.org,41143,1682614416328 in 182 msec 2023-04-27 16:54:02,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-04-27 16:54:02,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, ASSIGN in 343 msec 2023-04-27 16:54:02,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 in 559 msec 2023-04-27 16:54:03,026 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:54:03,030 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:03,032 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,032 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-27 16:54:03,036 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614443036"}]},"ts":"1682614443036"} 2023-04-27 16:54:03,037 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-04-27 16:54:03,136 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-04-27 16:54:03,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-27 16:54:03,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, UNASSIGN}] 2023-04-27 16:54:03,138 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, UNASSIGN 2023-04-27 16:54:03,138 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1b60f0ca3dad9cb4e6fbe55481d58af3, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:03,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614443138"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614443138"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614443138"}]},"ts":"1682614443138"} 2023-04-27 16:54:03,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 1b60f0ca3dad9cb4e6fbe55481d58af3, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:03,142 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, UNASSIGN 2023-04-27 16:54:03,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=bd98018c89359efe591534fa1b58d354, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:03,143 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614443143"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614443143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614443143"}]},"ts":"1682614443143"} 2023-04-27 16:54:03,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure bd98018c89359efe591534fa1b58d354, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:54:03,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:03,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing bd98018c89359efe591534fa1b58d354, disabling compactions & flushes 2023-04-27 16:54:03,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:03,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:03,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. after waiting 0 ms 2023-04-27 16:54:03,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:03,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-27 16:54:03,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:03,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354. 2023-04-27 16:54:03,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for bd98018c89359efe591534fa1b58d354: 2023-04-27 16:54:03,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:03,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:03,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 1b60f0ca3dad9cb4e6fbe55481d58af3, disabling compactions & flushes 2023-04-27 16:54:03,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:03,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:03,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. after waiting 0 ms 2023-04-27 16:54:03,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:03,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=bd98018c89359efe591534fa1b58d354, regionState=CLOSED 2023-04-27 16:54:03,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614443300"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614443300"}]},"ts":"1682614443300"} 2023-04-27 16:54:03,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-27 16:54:03,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-04-27 16:54:03,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure bd98018c89359efe591534fa1b58d354, server=jenkins-hbase12.apache.org,41143,1682614416328 in 158 msec 2023-04-27 16:54:03,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:03,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3. 2023-04-27 16:54:03,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 1b60f0ca3dad9cb4e6fbe55481d58af3: 2023-04-27 16:54:03,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=bd98018c89359efe591534fa1b58d354, UNASSIGN in 167 msec 2023-04-27 16:54:03,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:03,306 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1b60f0ca3dad9cb4e6fbe55481d58af3, regionState=CLOSED 2023-04-27 16:54:03,306 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614443306"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614443306"}]},"ts":"1682614443306"} 2023-04-27 16:54:03,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-04-27 16:54:03,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 1b60f0ca3dad9cb4e6fbe55481d58af3, server=jenkins-hbase12.apache.org,41143,1682614416328 in 167 msec 2023-04-27 16:54:03,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-04-27 16:54:03,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1b60f0ca3dad9cb4e6fbe55481d58af3, UNASSIGN in 172 msec 2023-04-27 16:54:03,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614443311"}]},"ts":"1682614443311"} 2023-04-27 16:54:03,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-04-27 16:54:03,324 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-04-27 16:54:03,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 293 msec 2023-04-27 16:54:03,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-27 16:54:03,338 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-04-27 16:54:03,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,342 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,343 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-04-27 16:54:03,347 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:03,347 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:03,347 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:03,349 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/recovered.edits] 2023-04-27 16:54:03,350 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/recovered.edits] 2023-04-27 16:54:03,350 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/recovered.edits] 2023-04-27 16:54:03,355 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/recovered.edits/7.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354/recovered.edits/7.seqid 2023-04-27 16:54:03,355 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/bd98018c89359efe591534fa1b58d354 2023-04-27 16:54:03,355 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/recovered.edits/4.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972/recovered.edits/4.seqid 2023-04-27 16:54:03,356 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/recovered.edits/7.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3/recovered.edits/7.seqid 2023-04-27 16:54:03,356 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/4da958a3a9419b0e6e2a5a7fefc50972 2023-04-27 16:54:03,356 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionNoStoreFile/1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:54:03,356 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-04-27 16:54:03,358 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,360 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-04-27 16:54:03,361 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-04-27 16:54:03,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-04-27 16:54:03,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614443363"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:03,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614443363"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:03,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614443363"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:03,365 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:54:03,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4da958a3a9419b0e6e2a5a7fefc50972, NAME => 'testSplitTableRegionNoStoreFile,,1682614441768.4da958a3a9419b0e6e2a5a7fefc50972.', STARTKEY => '', ENDKEY => ''}, {ENCODED => bd98018c89359efe591534fa1b58d354, NAME => 'testSplitTableRegionNoStoreFile,,1682614442271.bd98018c89359efe591534fa1b58d354.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 1b60f0ca3dad9cb4e6fbe55481d58af3, NAME => 'testSplitTableRegionNoStoreFile,41,1682614442271.1b60f0ca3dad9cb4e6fbe55481d58af3.', STARTKEY => '41', ENDKEY => ''}] 2023-04-27 16:54:03,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-04-27 16:54:03,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614443365"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:03,366 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-04-27 16:54:03,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:54:03,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 40 msec 2023-04-27 16:54:03,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-04-27 16:54:03,444 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-04-27 16:54:03,456 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402 (was 400) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1374923940_15 at /127.0.0.1:45428 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1374923940_15 at /127.0.0.1:37858 [Waiting for operation #43] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=838 (was 840), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=390 (was 415), ProcessCount=174 (was 174), AvailableMemoryMB=11775 (was 11779) 2023-04-27 16:54:03,468 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=390, ProcessCount=174, AvailableMemoryMB=11774 2023-04-27 16:54:03,469 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:03,482 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:54:03,482 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:54:03,484 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:03,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:54:03,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:54:03,488 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,489 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d empty. 2023-04-27 16:54:03,489 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-04-27 16:54:03,498 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-04-27 16:54:03,499 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => ca3d7a0582cb383ce3d284ca8b366c1d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:54:03,510 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:03,510 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing ca3d7a0582cb383ce3d284ca8b366c1d, disabling compactions & flushes 2023-04-27 16:54:03,510 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,510 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,510 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. after waiting 0 ms 2023-04-27 16:54:03,511 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,511 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,511 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:03,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:54:03,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614443514"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614443514"}]},"ts":"1682614443514"} 2023-04-27 16:54:03,515 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:54:03,516 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:54:03,516 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614443516"}]},"ts":"1682614443516"} 2023-04-27 16:54:03,517 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-04-27 16:54:03,536 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:54:03,537 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:54:03,537 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:54:03,537 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:54:03,537 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:54:03,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, ASSIGN}] 2023-04-27 16:54:03,539 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, ASSIGN 2023-04-27 16:54:03,540 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:54:03,690 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:54:03,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ca3d7a0582cb383ce3d284ca8b366c1d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:03,692 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614443692"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614443692"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614443692"}]},"ts":"1682614443692"} 2023-04-27 16:54:03,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure ca3d7a0582cb383ce3d284ca8b366c1d, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:03,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ca3d7a0582cb383ce3d284ca8b366c1d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:54:03,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:03,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:03,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,854 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,856 DEBUG [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1 2023-04-27 16:54:03,856 DEBUG [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1 2023-04-27 16:54:03,856 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ca3d7a0582cb383ce3d284ca8b366c1d columnFamilyName cf1 2023-04-27 16:54:03,857 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] regionserver.HStore(310): Store=ca3d7a0582cb383ce3d284ca8b366c1d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:03,857 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,858 DEBUG [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2 2023-04-27 16:54:03,858 DEBUG [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2 2023-04-27 16:54:03,859 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ca3d7a0582cb383ce3d284ca8b366c1d columnFamilyName cf2 2023-04-27 16:54:03,859 INFO [StoreOpener-ca3d7a0582cb383ce3d284ca8b366c1d-1] regionserver.HStore(310): Store=ca3d7a0582cb383ce3d284ca8b366c1d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:03,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:03,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:03,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:54:03,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened ca3d7a0582cb383ce3d284ca8b366c1d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65550217, jitterRate=-0.023225650191307068}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:03,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:03,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d., pid=71, masterSystemTime=1682614443845 2023-04-27 16:54:03,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:03,870 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ca3d7a0582cb383ce3d284ca8b366c1d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:03,870 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614443870"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614443870"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614443870"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614443870"}]},"ts":"1682614443870"} 2023-04-27 16:54:03,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-04-27 16:54:03,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure ca3d7a0582cb383ce3d284ca8b366c1d, server=jenkins-hbase12.apache.org,46011,1682614416374 in 179 msec 2023-04-27 16:54:03,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-04-27 16:54:03,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, ASSIGN in 335 msec 2023-04-27 16:54:03,875 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:54:03,876 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614443876"}]},"ts":"1682614443876"} 2023-04-27 16:54:03,877 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-04-27 16:54:03,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:54:03,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 475 msec 2023-04-27 16:54:03,994 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:03,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:03,996 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:03,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:03,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:03,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:03,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:03,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,008 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,008 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,008 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,008 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,009 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,009 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:04,011 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:04,011 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,011 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:04,010 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,011 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,011 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,011 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,012 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,012 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:54:04,012 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,012 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,020 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,020 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,020 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,020 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,021 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,021 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,031 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/888fb0ef448c497eaf0d8699c82c056a 2023-04-27 16:54:04,053 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/c4e755f84ea343e1a06aac7a9311bd7a 2023-04-27 16:54:04,057 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/888fb0ef448c497eaf0d8699c82c056a as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a 2023-04-27 16:54:04,061 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:54:04,062 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/c4e755f84ea343e1a06aac7a9311bd7a as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a 2023-04-27 16:54:04,066 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:54:04,067 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 55ms, sequenceid=6, compaction requested=false 2023-04-27 16:54:04,067 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,068 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:04,068 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,068 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:04,068 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:04,068 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,068 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,068 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,073 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,073 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,073 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,073 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@759ab5ba[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:04,075 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:04,075 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,083 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,083 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,083 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,084 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:04,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:04,084 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:04,094 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,094 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,094 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,104 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,104 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,104 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,104 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,104 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,104 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,105 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:04,108 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:04,108 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:04,108 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:04,108 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,108 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,115 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,116 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,116 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,116 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,118 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,118 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,157 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,157 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,157 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:04,157 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,158 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,258 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:04,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:04,275 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:04,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:04,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,277 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,277 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,277 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:04,277 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:04,277 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,277 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,359 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,360 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,360 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:04,360 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:04,360 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:04,361 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:54:04,369 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/d8cebe55b8544d9b9e18595cceb4445c 2023-04-27 16:54:04,388 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,389 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,389 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,389 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,389 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,389 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,391 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/5d072127b7c6464f88a050790945ee20 2023-04-27 16:54:04,395 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/d8cebe55b8544d9b9e18595cceb4445c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c 2023-04-27 16:54:04,400 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:54:04,401 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/5d072127b7c6464f88a050790945ee20 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20 2023-04-27 16:54:04,405 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:54:04,406 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 45ms, sequenceid=19, compaction requested=false 2023-04-27 16:54:04,406 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:04,406 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,406 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:04,406 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:04,406 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,406 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,406 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,419 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,419 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,419 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,419 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@736d87c4[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:04,422 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:04,422 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:04,431 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:04,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,432 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:04,441 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,441 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,441 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,441 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,442 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,441 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,452 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,452 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,452 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,452 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:04,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:04,455 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:04,455 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:04,455 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:04,455 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,455 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,462 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,463 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,463 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,463 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,463 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,465 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,465 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,465 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,474 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:04,473 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,474 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,574 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:04,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:04,590 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:04,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:04,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,592 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,592 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,592 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:04,592 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:04,593 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,593 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,599 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,599 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,599 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,600 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:04,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,600 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,601 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,601 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:04,601 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,601 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,601 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,601 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:54:04,610 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/13d6a1d167224fbe9da776811f3126e5 2023-04-27 16:54:04,622 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/78876a3fae9b431e80f7faf0a3d43205 2023-04-27 16:54:04,626 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/13d6a1d167224fbe9da776811f3126e5 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5 2023-04-27 16:54:04,631 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:54:04,631 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/78876a3fae9b431e80f7faf0a3d43205 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205 2023-04-27 16:54:04,636 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:54:04,636 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 35ms, sequenceid=32, compaction requested=true 2023-04-27 16:54:04,637 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:04,637 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,637 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:04,637 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:04,637 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,637 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,637 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,642 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,662 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,673 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,673 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:04,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45237c0b[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:04,675 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:04,676 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,683 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,683 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,683 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,684 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:04,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,684 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,684 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:04,694 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,694 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,694 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,694 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,704 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,704 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,704 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,715 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,715 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,715 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:04,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:04,718 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:04,718 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:04,718 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:04,718 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,718 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:04,725 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,725 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,725 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,725 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,725 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,726 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,726 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,727 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,728 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,729 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,729 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,736 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,736 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,736 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:04,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,736 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,737 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,837 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:04,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:04,851 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:04,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:04,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,852 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,852 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,853 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:04,853 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:04,853 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,853 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,862 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,862 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,863 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,864 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,865 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:04,865 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:04,865 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,865 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:04,865 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:54:04,873 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,874 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/badb4afbf5a142c884079734f5dd1617 2023-04-27 16:54:04,883 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,883 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,883 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,883 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,884 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,885 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/d3e54a95e472451db1f4b597a7129517 2023-04-27 16:54:04,889 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/badb4afbf5a142c884079734f5dd1617 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617 2023-04-27 16:54:04,892 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:54:04,893 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/d3e54a95e472451db1f4b597a7129517 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517 2023-04-27 16:54:04,897 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:54:04,898 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 33ms, sequenceid=45, compaction requested=true 2023-04-27 16:54:04,898 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:04,898 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:04,898 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:04,898 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:04,898 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,898 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:04,898 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:04,904 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,904 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,905 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,905 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@105b931e[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:04,906 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:04,906 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,915 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,915 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,915 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:04,916 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:04,925 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,925 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,925 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,966 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,966 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:04,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:04,970 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:04,970 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:04,970 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:04,970 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,970 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:04,990 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,990 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:04,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,991 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:04,992 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,992 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:04,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:04,999 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:04,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:04,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:04,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:04,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:04,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,000 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,001 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,100 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:05,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:05,115 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:05,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:05,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,117 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:05,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:05,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,117 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,125 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,125 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,125 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,125 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,126 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,126 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,127 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:05,128 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,128 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,128 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:05,128 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:54:05,136 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,142 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/2757eb8016de454faadb03c2783b8d24 2023-04-27 16:54:05,146 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,146 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,146 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,147 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,147 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,157 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/91eb48a722744385a48bb14db85d7219 2023-04-27 16:54:05,162 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/2757eb8016de454faadb03c2783b8d24 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24 2023-04-27 16:54:05,166 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:54:05,190 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/91eb48a722744385a48bb14db85d7219 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219 2023-04-27 16:54:05,194 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:54:05,195 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 67ms, sequenceid=58, compaction requested=true 2023-04-27 16:54:05,196 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:05,196 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,196 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:05,197 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:05,197 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,197 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,197 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,210 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,210 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,210 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,210 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:05,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@304cc3ca[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:05,212 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:05,212 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,220 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,221 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:05,220 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,221 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:05,231 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,231 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,231 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,241 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,241 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,241 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,241 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,241 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,242 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,242 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:05,244 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:05,244 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:05,245 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:05,245 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:05,245 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,245 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,252 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,252 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,253 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,253 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,255 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,293 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,294 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:05,293 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,293 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,294 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,394 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:05,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:05,409 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:05,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:05,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,411 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:05,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:05,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,411 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,431 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,432 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,432 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,433 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,433 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:05,433 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:05,433 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,433 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:05,433 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:54:05,441 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,441 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/48b0e28932164a53a28a32a490072c05 2023-04-27 16:54:05,452 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,452 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,452 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,452 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,452 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,453 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/2c33183ebed24683ae2f0df44a75959c 2023-04-27 16:54:05,457 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/48b0e28932164a53a28a32a490072c05 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05 2023-04-27 16:54:05,461 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:54:05,462 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/2c33183ebed24683ae2f0df44a75959c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c 2023-04-27 16:54:05,466 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:54:05,467 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 34ms, sequenceid=71, compaction requested=true 2023-04-27 16:54:05,467 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:05,467 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,467 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:05,467 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:05,467 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,467 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,467 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,473 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,473 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f8031a0[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:05,475 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:05,475 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,483 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,483 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,483 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,484 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,484 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:05,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,484 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,494 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,504 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,504 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,504 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,505 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:05,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:05,508 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:05,508 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:05,508 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:05,508 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,508 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,515 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,515 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,516 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,517 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,517 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,518 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,525 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,525 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,525 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,525 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,526 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,525 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,526 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,526 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,526 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,527 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,527 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,527 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,527 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,627 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:05,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:05,641 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:05,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:05,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,643 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,643 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,643 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:05,643 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:05,643 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,643 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,652 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,652 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,652 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,652 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,653 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,653 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,653 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,653 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,654 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,654 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,654 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,654 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-04-27 16:54:05,662 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,663 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:05,668 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:05,673 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,673 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,673 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,673 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,673 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,675 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:05,681 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:05,681 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/cb11e0bb58be4744b9d1c8c2598645a5 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:05,685 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:05,686 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5, entries=9, sequenceid=83, filesize=5.1 K 2023-04-27 16:54:05,686 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/fc42a2681f9046d3af6da8df4b726cd3 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:05,690 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:05,690 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3, entries=9, sequenceid=83, filesize=5.1 K 2023-04-27 16:54:05,691 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 37ms, sequenceid=83, compaction requested=true 2023-04-27 16:54:05,691 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:05,691 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,691 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:05,691 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:05,691 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,691 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,691 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,704 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,704 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,705 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,705 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:05,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@719dc37f[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:05,707 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:05,707 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,715 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,715 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:05,715 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,715 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:05,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:05,716 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:05,782 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,783 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,783 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,799 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,799 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,799 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,799 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,799 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,800 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,800 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:05,803 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:05,803 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:05,803 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:05,804 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,804 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,811 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,811 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,813 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,820 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,820 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:05,820 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:05,821 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,822 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,921 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:05,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:05,931 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:05,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:05,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,933 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:05,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:05,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:05,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,945 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,945 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,945 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,945 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:05,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:05,946 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,946 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,947 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:05,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,947 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:05,947 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,947 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,947 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-04-27 16:54:05,955 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/4160b546d22242699180c70d7c681415 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,957 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,958 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4160b546d22242699180c70d7c681415 2023-04-27 16:54:05,966 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:05,970 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:05,971 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/4160b546d22242699180c70d7c681415 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415 2023-04-27 16:54:05,975 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4160b546d22242699180c70d7c681415 2023-04-27 16:54:05,975 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415, entries=5, sequenceid=91, filesize=5.0 K 2023-04-27 16:54:05,976 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/47e989c9eeea4f4f86e93cef1f323fb9 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:05,979 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:05,980 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9, entries=5, sequenceid=91, filesize=5.0 K 2023-04-27 16:54:05,980 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 33ms, sequenceid=91, compaction requested=true 2023-04-27 16:54:05,981 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:05,981 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:05,981 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:05,981 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:05,981 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,981 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:05,981 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:05,988 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,988 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:05,989 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,989 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:05,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:05,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d926d31[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:05,991 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:05,991 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:05,999 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:05,999 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:05,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:05,999 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:05,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:06,000 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:06,010 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,010 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,010 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,020 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,020 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,020 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:06,020 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,020 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,021 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,021 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:06,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:06,023 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:06,023 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:06,023 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:06,023 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,023 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,031 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:06,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,032 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:06,033 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,033 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,034 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,139 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:06,139 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:06,139 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,139 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,140 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,240 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:06,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:06,249 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:54:06,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:54:06,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,251 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:06,251 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:06,251 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-27 16:54:06,251 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-27 16:54:06,251 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,251 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,273 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,273 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,273 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,274 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:54:06,275 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:54:06,275 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,275 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. started... 2023-04-27 16:54:06,275 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-04-27 16:54:06,283 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,283 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,283 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,284 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,284 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:06,284 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:06,290 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,295 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,303 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,307 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,308 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,312 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,312 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa, entries=5, sequenceid=99, filesize=5.0 K 2023-04-27 16:54:06,313 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/f0ed5818bbad42d38e49e0dc4f382f03 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,317 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,317 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03, entries=5, sequenceid=99, filesize=5.0 K 2023-04-27 16:54:06,318 INFO [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 43ms, sequenceid=99, compaction requested=true 2023-04-27 16:54:06,318 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:06,318 DEBUG [rs(jenkins-hbase12.apache.org,46011,1682614416374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,318 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:54:06,318 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:54:06,318 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,318 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-27 16:54:06,318 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-27 16:54:06,325 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,325 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:06,326 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,326 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-27 16:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b96bbda[Count = 0] remaining members to acquire global barrier 2023-04-27 16:54:06,328 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-27 16:54:06,328 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,336 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,336 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:54:06,336 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:06,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-27 16:54:06,337 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:54:06,346 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,346 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,346 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,347 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,347 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,346 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-27 16:54:06,347 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:06,346 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,347 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,347 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-27 16:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:06,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,46011,1682614416374': 2023-04-27 16:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:54:06,350 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-27 16:54:06,350 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:54:06,350 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:54:06,350 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,350 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:54:06,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,359 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:54:06,360 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,360 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,367 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,367 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,367 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:54:06,368 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-27 16:54:06,368 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,368 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:54:06,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:54:06,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:54:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,368 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:54:06,369 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,369 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:54:06,369 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:54:06,369 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,369 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:06,469 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:54:06,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:54:06,472 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,473 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:06,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:06,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:06,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:06,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, UNASSIGN}] 2023-04-27 16:54:06,476 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, UNASSIGN 2023-04-27 16:54:06,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ca3d7a0582cb383ce3d284ca8b366c1d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:06,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614446477"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614446477"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614446477"}]},"ts":"1682614446477"} 2023-04-27 16:54:06,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure ca3d7a0582cb383ce3d284ca8b366c1d, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:06,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing ca3d7a0582cb383ce3d284ca8b366c1d, disabling compactions & flushes 2023-04-27 16:54:06,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. after waiting 0 ms 2023-04-27 16:54:06,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing ca3d7a0582cb383ce3d284ca8b366c1d 2/2 column families, dataSize=50 B heapSize=720 B 2023-04-27 16:54:06,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf1/710812caa638473894edb4410ffddf91 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91, entries=1, sequenceid=103, filesize=4.9 K 2023-04-27 16:54:06,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.tmp/cf2/7e303d3147ec4f81bff96f9febeaa0bc as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,664 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,664 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc, entries=1, sequenceid=103, filesize=4.9 K 2023-04-27 16:54:06,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ca3d7a0582cb383ce3d284ca8b366c1d in 34ms, sequenceid=103, compaction requested=true 2023-04-27 16:54:06,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-04-27 16:54:06,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:06,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d. 2023-04-27 16:54:06,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for ca3d7a0582cb383ce3d284ca8b366c1d: 2023-04-27 16:54:06,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,672 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ca3d7a0582cb383ce3d284ca8b366c1d, regionState=CLOSED 2023-04-27 16:54:06,672 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614446672"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614446672"}]},"ts":"1682614446672"} 2023-04-27 16:54:06,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-04-27 16:54:06,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure ca3d7a0582cb383ce3d284ca8b366c1d, server=jenkins-hbase12.apache.org,46011,1682614416374 in 195 msec 2023-04-27 16:54:06,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-04-27 16:54:06,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ca3d7a0582cb383ce3d284ca8b366c1d, UNASSIGN in 199 msec 2023-04-27 16:54:06,689 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=ca3d7a0582cb383ce3d284ca8b366c1d, threads=16 2023-04-27 16:54:06,689 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,689 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,690 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,690 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,690 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,690 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,697 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,697 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,697 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,697 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,698 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,698 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,698 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,700 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,704 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,704 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,707 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24, top=false 2023-04-27 16:54:06,712 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:06,713 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,715 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,720 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:06,721 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,722 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,722 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517, top=false 2023-04-27 16:54:06,722 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:06,722 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc, top=true 2023-04-27 16:54:06,723 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219, top=false 2023-04-27 16:54:06,725 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c, top=false 2023-04-27 16:54:06,727 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05, top=false 2023-04-27 16:54:06,727 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 710812caa638473894edb4410ffddf91 2023-04-27 16:54:06,727 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91, top=true 2023-04-27 16:54:06,730 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:06,730 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4160b546d22242699180c70d7c681415 2023-04-27 16:54:06,730 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa, top=true 2023-04-27 16:54:06,751 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4160b546d22242699180c70d7c681415 2023-04-27 16:54:06,751 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415, top=true 2023-04-27 16:54:06,767 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,767 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,768 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,768 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,768 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05 for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,767 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24 for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,768 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,768 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,769 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:06,769 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9, top=true 2023-04-27 16:54:06,769 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:06,769 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03, top=true 2023-04-27 16:54:06,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,773 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,774 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,774 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617, top=false 2023-04-27 16:54:06,777 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:06,784 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219 for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,784 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517 for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,795 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,802 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415 for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,802 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,803 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,807 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,820 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:06,821 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,836 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617 for child: 758574340f27bbce277a4f2e49747b8e, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,836 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91 for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,836 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,840 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03 for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,841 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,842 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,851 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/.splits/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9 for child: a8a9f5a06f80540e69723d7fa4cd82c8, parent: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,851 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,888 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,890 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:06,905 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,915 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,920 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,935 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,940 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5 for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,951 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,952 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c for region: ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:06,952 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region ca3d7a0582cb383ce3d284ca8b366c1d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:54:06,995 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-04-27 16:54:06,997 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-04-27 16:54:06,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1682614446999"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1682614446999"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1682614446999"}]},"ts":"1682614446999"} 2023-04-27 16:54:06,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614446999"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614446999"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614446999"}]},"ts":"1682614446999"} 2023-04-27 16:54:06,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614446999"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614446999"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614446999"}]},"ts":"1682614446999"} 2023-04-27 16:54:07,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, ASSIGN}] 2023-04-27 16:54:07,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, ASSIGN 2023-04-27 16:54:07,005 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:54:07,006 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, ASSIGN 2023-04-27 16:54:07,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46011,1682614416374; forceNewPlan=false, retain=false 2023-04-27 16:54:07,156 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:54:07,157 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a8a9f5a06f80540e69723d7fa4cd82c8, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:07,157 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614447157"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614447157"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614447157"}]},"ts":"1682614447157"} 2023-04-27 16:54:07,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure a8a9f5a06f80540e69723d7fa4cd82c8, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:07,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=758574340f27bbce277a4f2e49747b8e, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:07,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614447161"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614447161"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614447161"}]},"ts":"1682614447161"} 2023-04-27 16:54:07,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 758574340f27bbce277a4f2e49747b8e, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:07,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a8a9f5a06f80540e69723d7fa4cd82c8, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.', STARTKEY => '60', ENDKEY => ''} 2023-04-27 16:54:07,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:07,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:07,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,316 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,317 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1 2023-04-27 16:54:07,317 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1 2023-04-27 16:54:07,318 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8a9f5a06f80540e69723d7fa4cd82c8 columnFamilyName cf1 2023-04-27 16:54:07,328 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-top 2023-04-27 16:54:07,332 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-top 2023-04-27 16:54:07,335 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:07,336 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-top 2023-04-27 16:54:07,340 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-top 2023-04-27 16:54:07,343 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:07,343 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:07,346 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4160b546d22242699180c70d7c681415 2023-04-27 16:54:07,346 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415 2023-04-27 16:54:07,350 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 710812caa638473894edb4410ffddf91 2023-04-27 16:54:07,350 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91 2023-04-27 16:54:07,350 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(310): Store=a8a9f5a06f80540e69723d7fa4cd82c8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:07,350 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,351 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2 2023-04-27 16:54:07,351 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2 2023-04-27 16:54:07,352 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8a9f5a06f80540e69723d7fa4cd82c8 columnFamilyName cf2 2023-04-27 16:54:07,363 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-top 2023-04-27 16:54:07,367 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-top 2023-04-27 16:54:07,371 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-top 2023-04-27 16:54:07,374 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:07,375 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-top 2023-04-27 16:54:07,379 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:07,379 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:07,383 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:07,383 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:07,386 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:07,386 DEBUG [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:07,386 INFO [StoreOpener-a8a9f5a06f80540e69723d7fa4cd82c8-1] regionserver.HStore(310): Store=a8a9f5a06f80540e69723d7fa4cd82c8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:07,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:07,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:07,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened a8a9f5a06f80540e69723d7fa4cd82c8; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61470762, jitterRate=-0.08401426672935486}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:07,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:07,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8., pid=77, masterSystemTime=1682614447310 2023-04-27 16:54:07,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,393 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:07,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,393 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:07,393 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,393 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,393 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): a8a9f5a06f80540e69723d7fa4cd82c8/cf1 is initiating minor compaction (all files) 2023-04-27 16:54:07,393 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): a8a9f5a06f80540e69723d7fa4cd82c8/cf2 is initiating minor compaction (all files) 2023-04-27 16:54:07,393 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a8a9f5a06f80540e69723d7fa4cd82c8/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,393 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a8a9f5a06f80540e69723d7fa4cd82c8/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,394 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp, totalSize=34.8 K 2023-04-27 16:54:07,394 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp, totalSize=34.8 K 2023-04-27 16:54:07,395 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614443987 2023-04-27 16:54:07,395 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614443987 2023-04-27 16:54:07,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 758574340f27bbce277a4f2e49747b8e, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.', STARTKEY => '', ENDKEY => '60'} 2023-04-27 16:54:07,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:07,396 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a8a9f5a06f80540e69723d7fa4cd82c8, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:07,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,396 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614444260 2023-04-27 16:54:07,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:07,396 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614447396"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614447396"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614447396"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614447396"}]},"ts":"1682614447396"} 2023-04-27 16:54:07,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,397 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614444260 2023-04-27 16:54:07,397 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614444576 2023-04-27 16:54:07,398 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614444576 2023-04-27 16:54:07,398 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1682614445628 2023-04-27 16:54:07,399 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,399 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,399 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1682614445628 2023-04-27 16:54:07,400 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,400 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,400 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,400 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1 2023-04-27 16:54:07,401 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1 2023-04-27 16:54:07,401 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-04-27 16:54:07,401 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 758574340f27bbce277a4f2e49747b8e columnFamilyName cf1 2023-04-27 16:54:07,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure a8a9f5a06f80540e69723d7fa4cd82c8, server=jenkins-hbase12.apache.org,46011,1682614416374 in 239 msec 2023-04-27 16:54:07,401 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, ASSIGN in 399 msec 2023-04-27 16:54:07,416 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-bottom 2023-04-27 16:54:07,426 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-bottom 2023-04-27 16:54:07,430 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): a8a9f5a06f80540e69723d7fa4cd82c8#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,430 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8a9f5a06f80540e69723d7fa4cd82c8#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,432 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:07,432 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-bottom 2023-04-27 16:54:07,436 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-bottom 2023-04-27 16:54:07,440 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24 2023-04-27 16:54:07,445 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05 2023-04-27 16:54:07,445 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d5712c3f89654329a003bc13d1f8c99e: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,445 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 13ed1eee891849c49a4943bdb37a568e: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,446 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp/cf1/13ed1eee891849c49a4943bdb37a568e as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13ed1eee891849c49a4943bdb37a568e 2023-04-27 16:54:07,446 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp/cf2/d5712c3f89654329a003bc13d1f8c99e as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/d5712c3f89654329a003bc13d1f8c99e 2023-04-27 16:54:07,449 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617 2023-04-27 16:54:07,449 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(310): Store=758574340f27bbce277a4f2e49747b8e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:07,449 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,450 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2 2023-04-27 16:54:07,450 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2 2023-04-27 16:54:07,450 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 13ed1eee891849c49a4943bdb37a568e: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,451 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 758574340f27bbce277a4f2e49747b8e columnFamilyName cf2 2023-04-27 16:54:07,451 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d5712c3f89654329a003bc13d1f8c99e: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,451 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8a9f5a06f80540e69723d7fa4cd82c8/cf2 of a8a9f5a06f80540e69723d7fa4cd82c8 into d5712c3f89654329a003bc13d1f8c99e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,451 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a8a9f5a06f80540e69723d7fa4cd82c8/cf1 of a8a9f5a06f80540e69723d7fa4cd82c8 into 13ed1eee891849c49a4943bdb37a568e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,451 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:07,451 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:07,451 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8., storeName=a8a9f5a06f80540e69723d7fa4cd82c8/cf2, priority=9, startTime=1682614447393; duration=0sec 2023-04-27 16:54:07,452 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8., storeName=a8a9f5a06f80540e69723d7fa4cd82c8/cf1, priority=9, startTime=1682614447392; duration=0sec 2023-04-27 16:54:07,452 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,452 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,460 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-bottom 2023-04-27 16:54:07,463 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-bottom 2023-04-27 16:54:07,466 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-bottom 2023-04-27 16:54:07,470 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:07,470 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-bottom 2023-04-27 16:54:07,473 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c 2023-04-27 16:54:07,476 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219 2023-04-27 16:54:07,479 DEBUG [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517 2023-04-27 16:54:07,479 INFO [StoreOpener-758574340f27bbce277a4f2e49747b8e-1] regionserver.HStore(310): Store=758574340f27bbce277a4f2e49747b8e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:07,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:54:07,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:07,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 758574340f27bbce277a4f2e49747b8e; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70151392, jitterRate=0.04533720016479492}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:54:07,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:07,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e., pid=78, masterSystemTime=1682614447310 2023-04-27 16:54:07,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:07,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:07,485 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:07,485 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:54:07,485 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,485 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,485 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1912): 758574340f27bbce277a4f2e49747b8e/cf1 is initiating minor compaction (all files) 2023-04-27 16:54:07,485 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1912): 758574340f27bbce277a4f2e49747b8e/cf2 is initiating minor compaction (all files) 2023-04-27 16:54:07,486 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 758574340f27bbce277a4f2e49747b8e/cf1 in testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,486 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 758574340f27bbce277a4f2e49747b8e/cf2 in testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,486 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp, totalSize=35.0 K 2023-04-27 16:54:07,486 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp, totalSize=35.0 K 2023-04-27 16:54:07,486 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614443987 2023-04-27 16:54:07,486 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614443987 2023-04-27 16:54:07,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,487 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614444260 2023-04-27 16:54:07,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=758574340f27bbce277a4f2e49747b8e, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:07,487 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614444260 2023-04-27 16:54:07,487 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614447487"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614447487"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614447487"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614447487"}]},"ts":"1682614447487"} 2023-04-27 16:54:07,487 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614444576 2023-04-27 16:54:07,487 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614444576 2023-04-27 16:54:07,487 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614444838 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614444838 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614445103 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614445103 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614445396 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614445396 2023-04-27 16:54:07,488 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614445628 2023-04-27 16:54:07,489 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614445628 2023-04-27 16:54:07,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-04-27 16:54:07,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 758574340f27bbce277a4f2e49747b8e, server=jenkins-hbase12.apache.org,46011,1682614416374 in 325 msec 2023-04-27 16:54:07,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-04-27 16:54:07,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, ASSIGN in 487 msec 2023-04-27 16:54:07,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 in 1.0200 sec 2023-04-27 16:54:07,501 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 758574340f27bbce277a4f2e49747b8e#cf2#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,508 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 758574340f27bbce277a4f2e49747b8e#cf1#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,524 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp/cf2/969b3b8826914a749c41e1b56fe94bf5 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/969b3b8826914a749c41e1b56fe94bf5 2023-04-27 16:54:07,530 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 758574340f27bbce277a4f2e49747b8e/cf2 of 758574340f27bbce277a4f2e49747b8e into 969b3b8826914a749c41e1b56fe94bf5(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,530 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:07,530 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e., storeName=758574340f27bbce277a4f2e49747b8e/cf2, priority=9, startTime=1682614447485; duration=0sec 2023-04-27 16:54:07,530 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,531 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b 2023-04-27 16:54:07,536 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 758574340f27bbce277a4f2e49747b8e/cf1 of 758574340f27bbce277a4f2e49747b8e into 5c5fe1aa3a8b4ad3a6d17dda6811a74b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,536 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:07,536 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e., storeName=758574340f27bbce277a4f2e49747b8e/cf1, priority=9, startTime=1682614447484; duration=0sec 2023-04-27 16:54:07,536 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,647 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:54:07,736 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-27 16:54:07,741 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:51910, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-27 16:54:07,742 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HStore(1912): 758574340f27bbce277a4f2e49747b8e/cf1 is initiating major compaction (all files) 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@944c9a7; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,743 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 758574340f27bbce277a4f2e49747b8e/cf1 in testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-27 16:54:07,743 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp, totalSize=6.8 K 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HStore(1912): 758574340f27bbce277a4f2e49747b8e/cf2 is initiating major compaction (all files) 2023-04-27 16:54:07,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f22ee64; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:07,743 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 758574340f27bbce277a4f2e49747b8e/cf2 in testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:07,743 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/969b3b8826914a749c41e1b56fe94bf5] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp, totalSize=6.8 K 2023-04-27 16:54:07,744 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 5c5fe1aa3a8b4ad3a6d17dda6811a74b, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614443987 2023-04-27 16:54:07,744 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting 969b3b8826914a749c41e1b56fe94bf5, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614443987 2023-04-27 16:54:07,745 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HStore(1912): a8a9f5a06f80540e69723d7fa4cd82c8/cf1 is initiating major compaction (all files) 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@66db1954; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HStore(1912): a8a9f5a06f80540e69723d7fa4cd82c8/cf2 is initiating major compaction (all files) 2023-04-27 16:54:07,746 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@126e693e; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-27 16:54:07,747 INFO [Listener at localhost.localdomain/45365] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-04-27 16:54:07,752 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 758574340f27bbce277a4f2e49747b8e#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,754 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): 758574340f27bbce277a4f2e49747b8e#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,770 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:07,771 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:07,773 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp/cf1/28e2b4423a274ffbb88fe3b875a1bce9 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/28e2b4423a274ffbb88fe3b875a1bce9 2023-04-27 16:54:07,775 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/.tmp/cf2/5cd42ad7d4a140fd8d05e9e20d9162d0 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5cd42ad7d4a140fd8d05e9e20d9162d0 2023-04-27 16:54:07,778 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 758574340f27bbce277a4f2e49747b8e/cf1 of 758574340f27bbce277a4f2e49747b8e into 28e2b4423a274ffbb88fe3b875a1bce9(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,778 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:07,779 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1682614447743; duration=0sec 2023-04-27 16:54:07,780 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-27 16:54:07,780 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a8a9f5a06f80540e69723d7fa4cd82c8/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,780 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13ed1eee891849c49a4943bdb37a568e] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp, totalSize=5.0 K 2023-04-27 16:54:07,780 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] compactions.Compactor(207): Compacting 13ed1eee891849c49a4943bdb37a568e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,781 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 758574340f27bbce277a4f2e49747b8e/cf2 of 758574340f27bbce277a4f2e49747b8e into 5cd42ad7d4a140fd8d05e9e20d9162d0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,781 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:07,782 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1682614447743; duration=0sec 2023-04-27 16:54:07,782 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:54:07,782 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a8a9f5a06f80540e69723d7fa4cd82c8/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:07,782 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/d5712c3f89654329a003bc13d1f8c99e] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp, totalSize=5.0 K 2023-04-27 16:54:07,782 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] compactions.Compactor(207): Compacting d5712c3f89654329a003bc13d1f8c99e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-27 16:54:07,784 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] throttle.PressureAwareThroughputController(145): a8a9f5a06f80540e69723d7fa4cd82c8#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,791 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] throttle.PressureAwareThroughputController(145): a8a9f5a06f80540e69723d7fa4cd82c8#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:54:07,805 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b0036a565c694a398628cf2b0782e865: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,805 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp/cf1/b0036a565c694a398628cf2b0782e865 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/b0036a565c694a398628cf2b0782e865 2023-04-27 16:54:07,809 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b0036a565c694a398628cf2b0782e865: NONE, but ROW specified in column family configuration 2023-04-27 16:54:07,810 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in a8a9f5a06f80540e69723d7fa4cd82c8/cf1 of a8a9f5a06f80540e69723d7fa4cd82c8 into b0036a565c694a398628cf2b0782e865(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:07,810 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:07,810 INFO [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1682614447746; duration=0sec 2023-04-27 16:54:07,811 DEBUG [RS:2;jenkins-hbase12:46011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:07,882 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:07,883 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:07,992 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:07,993 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:08,102 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:08,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:08,209 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e133e9ca814443c79e9d3417a4969d24: NONE, but ROW specified in column family configuration 2023-04-27 16:54:08,210 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/.tmp/cf2/e133e9ca814443c79e9d3417a4969d24 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/e133e9ca814443c79e9d3417a4969d24 2023-04-27 16:54:08,213 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:08,214 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:08,215 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e133e9ca814443c79e9d3417a4969d24: NONE, but ROW specified in column family configuration 2023-04-27 16:54:08,216 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in a8a9f5a06f80540e69723d7fa4cd82c8/cf2 of a8a9f5a06f80540e69723d7fa4cd82c8 into e133e9ca814443c79e9d3417a4969d24(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:54:08,216 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:08,216 INFO [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1682614447746; duration=0sec 2023-04-27 16:54:08,216 DEBUG [RS:2;jenkins-hbase12:46011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:54:08,323 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. is not mergeable because it has references 2023-04-27 16:54:08,324 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46011] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. is not mergeable because it has references 2023-04-27 16:54:08,331 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-04-27 16:54:08,332 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:08,334 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,334 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-27 16:54:08,338 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614448338"}]},"ts":"1682614448338"} 2023-04-27 16:54:08,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-04-27 16:54:08,399 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-04-27 16:54:08,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, UNASSIGN}] 2023-04-27 16:54:08,401 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, UNASSIGN 2023-04-27 16:54:08,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=a8a9f5a06f80540e69723d7fa4cd82c8, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:08,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614448401"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614448401"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614448401"}]},"ts":"1682614448401"} 2023-04-27 16:54:08,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure a8a9f5a06f80540e69723d7fa4cd82c8, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:08,405 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, UNASSIGN 2023-04-27 16:54:08,405 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=758574340f27bbce277a4f2e49747b8e, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:08,405 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614448405"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614448405"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614448405"}]},"ts":"1682614448405"} 2023-04-27 16:54:08,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 758574340f27bbce277a4f2e49747b8e, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:08,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-27 16:54:08,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:08,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing a8a9f5a06f80540e69723d7fa4cd82c8, disabling compactions & flushes 2023-04-27 16:54:08,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:08,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:08,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. after waiting 0 ms 2023-04-27 16:54:08,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:08,555 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13ed1eee891849c49a4943bdb37a568e, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91] to archive 2023-04-27 16:54:08,556 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:08,558 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,560 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,561 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,562 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,563 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-4160b546d22242699180c70d7c681415 2023-04-27 16:54:08,565 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:08,566 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13ed1eee891849c49a4943bdb37a568e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/13ed1eee891849c49a4943bdb37a568e 2023-04-27 16:54:08,567 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-710812caa638473894edb4410ffddf91 2023-04-27 16:54:08,568 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/d5712c3f89654329a003bc13d1f8c99e, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc] to archive 2023-04-27 16:54:08,569 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:08,570 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,571 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,572 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,573 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,574 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:08,575 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:08,576 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/d5712c3f89654329a003bc13d1f8c99e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/d5712c3f89654329a003bc13d1f8c99e 2023-04-27 16:54:08,577 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:08,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-04-27 16:54:08,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:08,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8. 2023-04-27 16:54:08,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for a8a9f5a06f80540e69723d7fa4cd82c8: 2023-04-27 16:54:08,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:08,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:08,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 758574340f27bbce277a4f2e49747b8e, disabling compactions & flushes 2023-04-27 16:54:08,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:08,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:08,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. after waiting 0 ms 2023-04-27 16:54:08,584 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=a8a9f5a06f80540e69723d7fa4cd82c8, regionState=CLOSED 2023-04-27 16:54:08,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:08,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614448584"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614448584"}]},"ts":"1682614448584"} 2023-04-27 16:54:08,585 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5-bottom] to archive 2023-04-27 16:54:08,586 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:08,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-04-27 16:54:08,587 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/888fb0ef448c497eaf0d8699c82c056a.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure a8a9f5a06f80540e69723d7fa4cd82c8, server=jenkins-hbase12.apache.org,46011,1682614416374 in 183 msec 2023-04-27 16:54:08,588 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/d8cebe55b8544d9b9e18595cceb4445c.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a8a9f5a06f80540e69723d7fa4cd82c8, UNASSIGN in 187 msec 2023-04-27 16:54:08,589 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/13d6a1d167224fbe9da776811f3126e5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,590 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-badb4afbf5a142c884079734f5dd1617 2023-04-27 16:54:08,591 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2757eb8016de454faadb03c2783b8d24 2023-04-27 16:54:08,592 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-48b0e28932164a53a28a32a490072c05 2023-04-27 16:54:08,593 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/5c5fe1aa3a8b4ad3a6d17dda6811a74b 2023-04-27 16:54:08,594 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/cb11e0bb58be4744b9d1c8c2598645a5.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,596 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/969b3b8826914a749c41e1b56fe94bf5, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3-bottom] to archive 2023-04-27 16:54:08,596 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:54:08,598 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/c4e755f84ea343e1a06aac7a9311bd7a.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,599 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5d072127b7c6464f88a050790945ee20.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,600 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/78876a3fae9b431e80f7faf0a3d43205.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,601 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-d3e54a95e472451db1f4b597a7129517 2023-04-27 16:54:08,602 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-91eb48a722744385a48bb14db85d7219 2023-04-27 16:54:08,603 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/testSplitTableRegionDeletedRowsDaughter=ca3d7a0582cb383ce3d284ca8b366c1d-2c33183ebed24683ae2f0df44a75959c 2023-04-27 16:54:08,603 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/969b3b8826914a749c41e1b56fe94bf5 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/969b3b8826914a749c41e1b56fe94bf5 2023-04-27 16:54:08,604 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/fc42a2681f9046d3af6da8df4b726cd3.ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-04-27 16:54:08,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:08,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e. 2023-04-27 16:54:08,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 758574340f27bbce277a4f2e49747b8e: 2023-04-27 16:54:08,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:08,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=758574340f27bbce277a4f2e49747b8e, regionState=CLOSED 2023-04-27 16:54:08,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1682614448611"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614448611"}]},"ts":"1682614448611"} 2023-04-27 16:54:08,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-04-27 16:54:08,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 758574340f27bbce277a4f2e49747b8e, server=jenkins-hbase12.apache.org,46011,1682614416374 in 206 msec 2023-04-27 16:54:08,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-04-27 16:54:08,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=758574340f27bbce277a4f2e49747b8e, UNASSIGN in 214 msec 2023-04-27 16:54:08,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614448616"}]},"ts":"1682614448616"} 2023-04-27 16:54:08,617 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-04-27 16:54:08,629 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-04-27 16:54:08,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 295 msec 2023-04-27 16:54:08,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-27 16:54:08,641 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-04-27 16:54:08,642 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,645 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-04-27 16:54:08,650 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,650 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:08,650 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:08,651 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/recovered.edits] 2023-04-27 16:54:08,651 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/recovered.edits] 2023-04-27 16:54:08,651 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/recovered.edits] 2023-04-27 16:54:08,654 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/28e2b4423a274ffbb88fe3b875a1bce9 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf1/28e2b4423a274ffbb88fe3b875a1bce9 2023-04-27 16:54:08,654 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/b0036a565c694a398628cf2b0782e865 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf1/b0036a565c694a398628cf2b0782e865 2023-04-27 16:54:08,654 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/0942d9fdfbdc4f39ada33e39cd2d49fa 2023-04-27 16:54:08,655 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/13d6a1d167224fbe9da776811f3126e5 2023-04-27 16:54:08,656 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/e133e9ca814443c79e9d3417a4969d24 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/cf2/e133e9ca814443c79e9d3417a4969d24 2023-04-27 16:54:08,657 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5cd42ad7d4a140fd8d05e9e20d9162d0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/cf2/5cd42ad7d4a140fd8d05e9e20d9162d0 2023-04-27 16:54:08,657 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/2757eb8016de454faadb03c2783b8d24 2023-04-27 16:54:08,658 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/4160b546d22242699180c70d7c681415 2023-04-27 16:54:08,659 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/48b0e28932164a53a28a32a490072c05 2023-04-27 16:54:08,659 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/recovered.edits/113.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e/recovered.edits/113.seqid 2023-04-27 16:54:08,659 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/recovered.edits/113.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8/recovered.edits/113.seqid 2023-04-27 16:54:08,660 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/758574340f27bbce277a4f2e49747b8e 2023-04-27 16:54:08,660 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:54:08,660 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/710812caa638473894edb4410ffddf91 2023-04-27 16:54:08,661 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/888fb0ef448c497eaf0d8699c82c056a 2023-04-27 16:54:08,662 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/badb4afbf5a142c884079734f5dd1617 2023-04-27 16:54:08,663 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/cb11e0bb58be4744b9d1c8c2598645a5 2023-04-27 16:54:08,664 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf1/d8cebe55b8544d9b9e18595cceb4445c 2023-04-27 16:54:08,667 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/2c33183ebed24683ae2f0df44a75959c 2023-04-27 16:54:08,668 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/47e989c9eeea4f4f86e93cef1f323fb9 2023-04-27 16:54:08,670 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/5d072127b7c6464f88a050790945ee20 2023-04-27 16:54:08,671 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/78876a3fae9b431e80f7faf0a3d43205 2023-04-27 16:54:08,672 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/7e303d3147ec4f81bff96f9febeaa0bc 2023-04-27 16:54:08,673 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/91eb48a722744385a48bb14db85d7219 2023-04-27 16:54:08,674 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/c4e755f84ea343e1a06aac7a9311bd7a 2023-04-27 16:54:08,675 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/d3e54a95e472451db1f4b597a7129517 2023-04-27 16:54:08,676 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/f0ed5818bbad42d38e49e0dc4f382f03 2023-04-27 16:54:08,677 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/cf2/fc42a2681f9046d3af6da8df4b726cd3 2023-04-27 16:54:08,679 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/recovered.edits/106.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d/recovered.edits/106.seqid 2023-04-27 16:54:08,679 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ca3d7a0582cb383ce3d284ca8b366c1d 2023-04-27 16:54:08,679 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-04-27 16:54:08,681 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,684 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-04-27 16:54:08,686 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-04-27 16:54:08,687 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,687 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-04-27 16:54:08,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614448687"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:08,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614448687"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:08,688 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614448687"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:08,690 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:54:08,690 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ca3d7a0582cb383ce3d284ca8b366c1d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682614443483.ca3d7a0582cb383ce3d284ca8b366c1d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 758574340f27bbce277a4f2e49747b8e, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1682614446472.758574340f27bbce277a4f2e49747b8e.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => a8a9f5a06f80540e69723d7fa4cd82c8, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1682614446472.a8a9f5a06f80540e69723d7fa4cd82c8.', STARTKEY => '60', ENDKEY => ''}] 2023-04-27 16:54:08,690 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-04-27 16:54:08,690 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614448690"}]},"ts":"9223372036854775807"} 2023-04-27 16:54:08,692 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-04-27 16:54:08,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:54:08,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 63 msec 2023-04-27 16:54:08,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-04-27 16:54:08,746 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-04-27 16:54:08,759 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=410 (was 402) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60976 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39546 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60944 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39180 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60706 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60960 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:60646 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39168 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39598 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39110 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39142 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39154 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ad4950d-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39132 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39194 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39570 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1795103418_15 at /127.0.0.1:39534 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d42571f-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=847 (was 838) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=375 (was 390), ProcessCount=173 (was 174), AvailableMemoryMB=11713 (was 11774) 2023-04-27 16:54:08,772 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=410, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=375, ProcessCount=173, AvailableMemoryMB=11712 2023-04-27 16:54:08,772 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:54:08,778 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:54:08,778 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:54:08,780 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:54:08,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:54:08,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:54:08,784 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:08,784 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 empty. 2023-04-27 16:54:08,784 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:08,785 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-04-27 16:54:08,797 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-04-27 16:54:08,798 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 36ff433ee5b46c5cccef4e1cd0657044, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:54:08,806 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:08,806 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 36ff433ee5b46c5cccef4e1cd0657044, disabling compactions & flushes 2023-04-27 16:54:08,806 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:08,806 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:08,807 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. after waiting 0 ms 2023-04-27 16:54:08,807 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:08,807 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:08,807 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 36ff433ee5b46c5cccef4e1cd0657044: 2023-04-27 16:54:08,808 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:54:08,809 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614448809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614448809"}]},"ts":"1682614448809"} 2023-04-27 16:54:08,810 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:54:08,811 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:54:08,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614448811"}]},"ts":"1682614448811"} 2023-04-27 16:54:08,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-04-27 16:54:08,871 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:54:08,872 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:54:08,872 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:54:08,872 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:54:08,872 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:54:08,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, ASSIGN}] 2023-04-27 16:54:08,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, ASSIGN 2023-04-27 16:54:08,875 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=false 2023-04-27 16:54:09,025 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:54:09,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449027"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614449027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614449027"}]},"ts":"1682614449027"} 2023-04-27 16:54:09,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:54:09,182 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,182 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-27 16:54:09,185 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:41188, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-27 16:54:09,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36ff433ee5b46c5cccef4e1cd0657044, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:54:09,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:09,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:09,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,192 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,193 DEBUG [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:54:09,193 DEBUG [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:54:09,193 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36ff433ee5b46c5cccef4e1cd0657044 columnFamilyName cf1 2023-04-27 16:54:09,194 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] regionserver.HStore(310): Store=36ff433ee5b46c5cccef4e1cd0657044/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:09,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:54:09,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 36ff433ee5b46c5cccef4e1cd0657044; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64892835, jitterRate=-0.03302140533924103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-27 16:54:09,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 36ff433ee5b46c5cccef4e1cd0657044: 2023-04-27 16:54:09,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044., pid=87, masterSystemTime=1682614449182 2023-04-27 16:54:09,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,205 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449205"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614449205"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614449205"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614449205"}]},"ts":"1682614449205"} 2023-04-27 16:54:09,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-04-27 16:54:09,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269 in 177 msec 2023-04-27 16:54:09,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-04-27 16:54:09,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, ASSIGN in 336 msec 2023-04-27 16:54:09,210 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:54:09,210 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614449210"}]},"ts":"1682614449210"} 2023-04-27 16:54:09,211 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-04-27 16:54:09,263 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:54:09,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 484 msec 2023-04-27 16:54:09,290 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$12(2454): Client=jenkins//136.243.104.168 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-27 16:54:09,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:54:09,313 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-04-27 16:54:09,313 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-04-27 16:54:09,313 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-04-27 16:54:09,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-04-27 16:54:09,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, REOPEN/MOVE}] 2023-04-27 16:54:09,330 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, REOPEN/MOVE 2023-04-27 16:54:09,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449330"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614449330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614449330"}]},"ts":"1682614449330"} 2023-04-27 16:54:09,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:54:09,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 36ff433ee5b46c5cccef4e1cd0657044, disabling compactions & flushes 2023-04-27 16:54:09,485 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. after waiting 0 ms 2023-04-27 16:54:09,485 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-04-27 16:54:09,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:54:09,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 36ff433ee5b46c5cccef4e1cd0657044: 2023-04-27 16:54:09,490 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 36ff433ee5b46c5cccef4e1cd0657044 to self. 2023-04-27 16:54:09,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,491 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=CLOSED 2023-04-27 16:54:09,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449491"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614449491"}]},"ts":"1682614449491"} 2023-04-27 16:54:09,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-04-27 16:54:09,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269 in 161 msec 2023-04-27 16:54:09,495 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=true 2023-04-27 16:54:09,645 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:54:09,645 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,645 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449645"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614449645"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614449645"}]},"ts":"1682614449645"} 2023-04-27 16:54:09,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:54:09,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36ff433ee5b46c5cccef4e1cd0657044, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:54:09,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:09,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:09,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,804 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,804 DEBUG [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:54:09,804 DEBUG [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:54:09,805 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36ff433ee5b46c5cccef4e1cd0657044 columnFamilyName cf1 2023-04-27 16:54:09,805 INFO [StoreOpener-36ff433ee5b46c5cccef4e1cd0657044-1] regionserver.HStore(310): Store=36ff433ee5b46c5cccef4e1cd0657044/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:54:09,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:54:09,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 36ff433ee5b46c5cccef4e1cd0657044; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72025931, jitterRate=0.07327000796794891}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-27 16:54:09,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 36ff433ee5b46c5cccef4e1cd0657044: 2023-04-27 16:54:09,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044., pid=92, masterSystemTime=1682614449799 2023-04-27 16:54:09,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:54:09,812 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:54:09,813 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449812"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614449812"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614449812"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614449812"}]},"ts":"1682614449812"} 2023-04-27 16:54:09,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-04-27 16:54:09,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269 in 167 msec 2023-04-27 16:54:09,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-04-27 16:54:09,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, REOPEN/MOVE in 487 msec 2023-04-27 16:54:09,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-04-27 16:54:09,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 502 msec 2023-04-27 16:54:09,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, ASSIGN}] 2023-04-27 16:54:09,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, ASSIGN 2023-04-27 16:54:09,824 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-27 16:54:09,974 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:54:09,975 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:54:09,975 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:54:09,975 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:54:09,975 DEBUG [jenkins-hbase12:37667] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-04-27 16:54:09,977 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=45bbccc83e9d10da908026afa9931611, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:54:09,977 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614449977"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1682614449977"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1682614449977"}]},"ts":"1682614449977"} 2023-04-27 16:54:09,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 45bbccc83e9d10da908026afa9931611, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:54:10,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 45bbccc83e9d10da908026afa9931611, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-04-27 16:54:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 45bbccc83e9d10da908026afa9931611 2023-04-27 16:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 45bbccc83e9d10da908026afa9931611 2023-04-27 16:54:10,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 45bbccc83e9d10da908026afa9931611 2023-04-27 16:54:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:10,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:11,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,817 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:54:12,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:12,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:13,958 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-04-27 16:54:14,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:14,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:15,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:16,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:17,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:18,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:19,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:20,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:21,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:22,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:23,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:24,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:25,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:26,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:27,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:28,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:29,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:30,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:31,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:32,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:33,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:34,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:35,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:36,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:37,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:38,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:39,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:40,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:41,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:42,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:43,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:44,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:45,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:46,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:47,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:48,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:49,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:50,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:51,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:52,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:53,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:54,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:55,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:56,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:57,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:58,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:54:59,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:00,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:01,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:02,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:03,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:04,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:05,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:06,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:07,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:08,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-27 16:55:09,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-27 16:55:09,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-27 16:55:09,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:09,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-27 16:55:10,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-27 16:55:10,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:10,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,511 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:11,512 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:11,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:11,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:11,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:11,515 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 36ff433ee5b46c5cccef4e1cd0657044, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:11,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 exec-time=6 msec 2023-04-27 16:55:11,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-27 16:55:11,861 INFO [StoreOpener-45bbccc83e9d10da908026afa9931611-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 45bbccc83e9d10da908026afa9931611 2023-04-27 16:55:11,864 DEBUG [StoreOpener-45bbccc83e9d10da908026afa9931611-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:55:11,864 DEBUG [StoreOpener-45bbccc83e9d10da908026afa9931611-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1 2023-04-27 16:55:11,864 INFO [StoreOpener-45bbccc83e9d10da908026afa9931611-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45bbccc83e9d10da908026afa9931611 columnFamilyName cf1 2023-04-27 16:55:11,866 INFO [StoreOpener-45bbccc83e9d10da908026afa9931611-1] regionserver.HStore(310): Store=45bbccc83e9d10da908026afa9931611/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:11,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 45bbccc83e9d10da908026afa9931611; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59786138, jitterRate=-0.10911712050437927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-27 16:55:11,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 45bbccc83e9d10da908026afa9931611: 2023-04-27 16:55:11,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611., pid=94, masterSystemTime=1682614450133 2023-04-27 16:55:11,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:11,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:11,870 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=45bbccc83e9d10da908026afa9931611, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:11,870 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614511870"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1682614511870"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1682614511870"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1682614511870"}]},"ts":"1682614511870"} 2023-04-27 16:55:11,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-04-27 16:55:11,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 45bbccc83e9d10da908026afa9931611, server=jenkins-hbase12.apache.org,46011,1682614416374 in 1 mins, 1.893 sec 2023-04-27 16:55:11,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-04-27 16:55:11,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, ASSIGN in 1 mins, 2.051 sec 2023-04-27 16:55:11,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.582 sec 2023-04-27 16:55:11,877 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1682614449287","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62589,"client":"136.243.104.168:45616","queuetimems":0,"class":"HMaster"} 2023-04-27 16:55:11,877 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 647, skipping over this response of 3 bytes 2023-04-27 16:55:13,763 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up 2023-04-27 16:55:13,765 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:13,766 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:13,766 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:13,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:13,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-27 16:55:13,770 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614513770"}]},"ts":"1682614513770"} 2023-04-27 16:55:13,771 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-04-27 16:55:13,838 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-04-27 16:55:13,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, UNASSIGN}] 2023-04-27 16:55:13,840 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, UNASSIGN 2023-04-27 16:55:13,840 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=45bbccc83e9d10da908026afa9931611, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:13,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614513840"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1682614513840"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1682614513840"}]},"ts":"1682614513840"} 2023-04-27 16:55:13,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 45bbccc83e9d10da908026afa9931611, server=jenkins-hbase12.apache.org,46011,1682614416374}] 2023-04-27 16:55:13,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, UNASSIGN 2023-04-27 16:55:13,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:13,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614513846"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614513846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614513846"}]},"ts":"1682614513846"} 2023-04-27 16:55:13,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:13,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-27 16:55:13,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 45bbccc83e9d10da908026afa9931611 2023-04-27 16:55:13,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 45bbccc83e9d10da908026afa9931611, disabling compactions & flushes 2023-04-27 16:55:13,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:13,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:13,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. after waiting 0 ms 2023-04-27 16:55:13,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:13,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:13,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611. 2023-04-27 16:55:13,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 45bbccc83e9d10da908026afa9931611: 2023-04-27 16:55:13,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 45bbccc83e9d10da908026afa9931611 2023-04-27 16:55:13,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=45bbccc83e9d10da908026afa9931611, regionState=CLOSED 2023-04-27 16:55:14,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614513999"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1682614513999"}]},"ts":"1682614513999"} 2023-04-27 16:55:14,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:55:14,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 36ff433ee5b46c5cccef4e1cd0657044, disabling compactions & flushes 2023-04-27 16:55:14,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:55:14,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:55:14,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. after waiting 0 ms 2023-04-27 16:55:14,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:55:14,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-04-27 16:55:14,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 45bbccc83e9d10da908026afa9931611, server=jenkins-hbase12.apache.org,46011,1682614416374 in 160 msec 2023-04-27 16:55:14,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-27 16:55:14,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611, UNASSIGN in 166 msec 2023-04-27 16:55:14,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:14,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044. 2023-04-27 16:55:14,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 36ff433ee5b46c5cccef4e1cd0657044: 2023-04-27 16:55:14,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:55:14,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=36ff433ee5b46c5cccef4e1cd0657044, regionState=CLOSED 2023-04-27 16:55:14,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1682614514009"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614514009"}]},"ts":"1682614514009"} 2023-04-27 16:55:14,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-04-27 16:55:14,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 36ff433ee5b46c5cccef4e1cd0657044, server=jenkins-hbase12.apache.org,44953,1682614416269 in 163 msec 2023-04-27 16:55:14,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-04-27 16:55:14,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=36ff433ee5b46c5cccef4e1cd0657044, UNASSIGN in 174 msec 2023-04-27 16:55:14,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614514018"}]},"ts":"1682614514018"} 2023-04-27 16:55:14,019 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-04-27 16:55:14,027 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-04-27 16:55:14,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 261 msec 2023-04-27 16:55:14,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-27 16:55:14,072 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-04-27 16:55:14,073 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-04-27 16:55:14,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,081 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:55:14,082 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits] 2023-04-27 16:55:14,086 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits/7.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044/recovered.edits/7.seqid 2023-04-27 16:55:14,086 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/36ff433ee5b46c5cccef4e1cd0657044 2023-04-27 16:55:14,086 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-04-27 16:55:14,088 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,090 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-04-27 16:55:14,092 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-04-27 16:55:14,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-04-27 16:55:14,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614514093"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:14,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614514093"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:14,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-04-27 16:55:14,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 36ff433ee5b46c5cccef4e1cd0657044, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779.36ff433ee5b46c5cccef4e1cd0657044.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 45bbccc83e9d10da908026afa9931611, NAME => 'testRollbackForSplitTableRegionWithReplica,,1682614448779_0001.45bbccc83e9d10da908026afa9931611.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-04-27 16:55:14,095 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-04-27 16:55:14,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614514095"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:14,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-04-27 16:55:14,109 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:14,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 36 msec 2023-04-27 16:55:14,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-04-27 16:55:14,178 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-04-27 16:55:14,190 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=345 (was 410), OpenFileDescriptor=654 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 375), ProcessCount=171 (was 173), AvailableMemoryMB=11714 (was 11712) - AvailableMemoryMB LEAK? - 2023-04-27 16:55:14,201 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=346, OpenFileDescriptor=654, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=171, AvailableMemoryMB=11714 2023-04-27 16:55:14,202 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:14,214 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:55:14,214 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:55:14,215 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:14,217 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:55:14,218 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:55:14,219 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,220 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef empty. 2023-04-27 16:55:14,220 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-04-27 16:55:14,229 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-04-27 16:55:14,230 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => c38782625824b2e26f5fb73831d5eaef, NAME => 'testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing c38782625824b2e26f5fb73831d5eaef, disabling compactions & flushes 2023-04-27 16:55:14,237 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. after waiting 0 ms 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,237 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,237 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:14,239 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:55:14,240 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682614514239"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614514239"}]},"ts":"1682614514239"} 2023-04-27 16:55:14,241 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:55:14,241 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:55:14,241 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614514241"}]},"ts":"1682614514241"} 2023-04-27 16:55:14,242 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-04-27 16:55:14,259 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:55:14,260 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:55:14,260 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:55:14,260 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:55:14,260 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:55:14,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, ASSIGN}] 2023-04-27 16:55:14,262 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, ASSIGN 2023-04-27 16:55:14,262 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=false 2023-04-27 16:55:14,412 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:55:14,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c38782625824b2e26f5fb73831d5eaef, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682614514413"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614514413"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614514413"}]},"ts":"1682614514413"} 2023-04-27 16:55:14,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure c38782625824b2e26f5fb73831d5eaef, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:14,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c38782625824b2e26f5fb73831d5eaef, NAME => 'testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:55:14,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:14,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:14,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,572 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,573 DEBUG [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1 2023-04-27 16:55:14,573 DEBUG [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1 2023-04-27 16:55:14,574 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c38782625824b2e26f5fb73831d5eaef columnFamilyName cf1 2023-04-27 16:55:14,575 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] regionserver.HStore(310): Store=c38782625824b2e26f5fb73831d5eaef/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:14,575 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,576 DEBUG [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2 2023-04-27 16:55:14,576 DEBUG [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2 2023-04-27 16:55:14,576 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c38782625824b2e26f5fb73831d5eaef columnFamilyName cf2 2023-04-27 16:55:14,577 INFO [StoreOpener-c38782625824b2e26f5fb73831d5eaef-1] regionserver.HStore(310): Store=c38782625824b2e26f5fb73831d5eaef/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:14,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:14,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:14,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:55:14,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened c38782625824b2e26f5fb73831d5eaef; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71274124, jitterRate=0.06206721067428589}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:14,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:14,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef., pid=104, masterSystemTime=1682614514567 2023-04-27 16:55:14,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,585 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c38782625824b2e26f5fb73831d5eaef, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,585 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682614514585"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614514585"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614514585"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614514585"}]},"ts":"1682614514585"} 2023-04-27 16:55:14,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-04-27 16:55:14,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure c38782625824b2e26f5fb73831d5eaef, server=jenkins-hbase12.apache.org,44953,1682614416269 in 173 msec 2023-04-27 16:55:14,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-04-27 16:55:14,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, ASSIGN in 330 msec 2023-04-27 16:55:14,593 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:55:14,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614514593"}]},"ts":"1682614514593"} 2023-04-27 16:55:14,594 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-04-27 16:55:14,607 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:55:14,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 392 msec 2023-04-27 16:55:14,718 DEBUG [Listener at localhost.localdomain/45365] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-27 16:55:14,722 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:40346, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-27 16:55:14,727 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:14,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:14,731 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:14,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:14,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:14,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:14,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,743 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,743 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:14,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:14,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:14,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:14,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:14,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:14,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:14,745 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,745 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:14,746 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:55:14,753 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,762 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/ad2514b75714481dbd232af559ac5212 2023-04-27 16:55:14,764 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,764 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,764 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:14,764 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,764 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:14,775 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/51372a956ce741bdbe44667af9ecc403 2023-04-27 16:55:14,779 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/ad2514b75714481dbd232af559ac5212 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/ad2514b75714481dbd232af559ac5212 2023-04-27 16:55:14,783 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/ad2514b75714481dbd232af559ac5212, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:14,783 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/51372a956ce741bdbe44667af9ecc403 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/51372a956ce741bdbe44667af9ecc403 2023-04-27 16:55:14,787 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/51372a956ce741bdbe44667af9ecc403, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:14,788 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 42ms, sequenceid=6, compaction requested=false 2023-04-27 16:55:14,788 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-04-27 16:55:14,789 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:14,789 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:14,789 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:14,789 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:14,789 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,789 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:14,789 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:14,795 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,796 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:14,796 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,796 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:14,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:14,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45359a39[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:14,798 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:14,798 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,806 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,807 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:14,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:14,807 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:14,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:14,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:14,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:14,827 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:14,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:14,831 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:14,831 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:14,831 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:14,831 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:14,831 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:14,838 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,839 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,839 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:14,840 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,840 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,841 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,848 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,848 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:14,849 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:14,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:14,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:14,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:14,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:14,849 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:14,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:14,850 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:14,850 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,950 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:14,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:14,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:14,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:14,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:14,964 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:14,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:14,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:14,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:14,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:14,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,007 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,007 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,007 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:15,009 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:15,009 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,009 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:15,009 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:15,018 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/257a38b3ec084840af99f9a102fe86f1 2023-04-27 16:55:15,027 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,027 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,027 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,027 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,027 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,028 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,029 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/f57bf8418583423db631f090bc10d010 2023-04-27 16:55:15,033 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/257a38b3ec084840af99f9a102fe86f1 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1 2023-04-27 16:55:15,037 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:15,037 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/f57bf8418583423db631f090bc10d010 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010 2023-04-27 16:55:15,041 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:15,042 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 33ms, sequenceid=19, compaction requested=false 2023-04-27 16:55:15,042 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:15,042 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:15,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:15,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,042 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,048 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:15,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a2c2fdb[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:15,051 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:15,051 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:15,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,059 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,059 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:15,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,080 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,080 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,080 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,080 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,090 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,090 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,090 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:15,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:15,093 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:15,093 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:15,093 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:15,093 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:15,093 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,101 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,103 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,103 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,104 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,111 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:15,112 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,111 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:15,111 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,112 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,113 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,212 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:15,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:15,226 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:15,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:15,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:15,227 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:15,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:15,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:15,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,227 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,238 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,238 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,238 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,238 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:15,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,239 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,239 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,240 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,240 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,240 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:15,240 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:15,248 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/a498750b57784238946c983eaf5f1044 2023-04-27 16:55:15,248 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,259 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/88de9cb62eb242b69207ecfdb14bdb05 2023-04-27 16:55:15,263 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/a498750b57784238946c983eaf5f1044 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/a498750b57784238946c983eaf5f1044 2023-04-27 16:55:15,266 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/a498750b57784238946c983eaf5f1044, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:15,267 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/88de9cb62eb242b69207ecfdb14bdb05 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/88de9cb62eb242b69207ecfdb14bdb05 2023-04-27 16:55:15,271 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/88de9cb62eb242b69207ecfdb14bdb05, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:15,271 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 31ms, sequenceid=32, compaction requested=true 2023-04-27 16:55:15,271 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:15,271 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,271 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:15,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:15,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,280 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,280 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,280 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,281 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:15,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74f92a4a[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:15,283 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:15,283 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,290 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,290 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,290 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:15,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,291 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,291 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:15,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,311 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,311 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,311 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,311 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,311 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,312 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:15,315 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:15,315 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:15,315 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:15,315 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:15,315 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,323 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,323 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,325 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,325 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,325 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,358 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:15,358 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:15,358 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:15,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,357 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:15,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,358 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,359 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,458 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:15,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:15,472 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:15,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:15,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:15,473 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:15,473 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,473 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:15,474 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:15,474 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,474 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,532 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,532 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,532 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,533 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:15,534 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:15,534 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,534 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:15,534 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:15,545 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/e38b95abd6a24bdb83980529801e4857 2023-04-27 16:55:15,557 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/5afc18669819444ebad25ceaf5615689 2023-04-27 16:55:15,561 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/e38b95abd6a24bdb83980529801e4857 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/e38b95abd6a24bdb83980529801e4857 2023-04-27 16:55:15,564 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/e38b95abd6a24bdb83980529801e4857, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:15,565 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/5afc18669819444ebad25ceaf5615689 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/5afc18669819444ebad25ceaf5615689 2023-04-27 16:55:15,568 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/5afc18669819444ebad25ceaf5615689, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:15,569 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 35ms, sequenceid=45, compaction requested=true 2023-04-27 16:55:15,569 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:15,569 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,569 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:15,569 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:15,569 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,569 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,569 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,596 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,596 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,596 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,606 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,606 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,606 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,607 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,607 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:15,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aef1764[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:15,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:15,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:15,617 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,617 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:15,627 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,627 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,627 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,638 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:15,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,638 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:15,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:15,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:15,641 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:15,641 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:15,641 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:15,641 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:15,641 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,648 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,649 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,649 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,650 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,651 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,651 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,651 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,659 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:15,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:15,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,659 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:15,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,660 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,760 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:15,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:15,774 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:15,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:15,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:15,776 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:15,776 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,776 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:15,776 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:15,776 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,776 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,785 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:15,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,786 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,786 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:15,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:15,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,788 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,787 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:15,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:15,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:15,788 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,788 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:15,788 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:15,795 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,796 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/286cf04964004cdfb476199fa40a3fee 2023-04-27 16:55:15,806 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,806 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,806 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,806 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,806 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,806 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/b28132f0b73148bca224f0a96c531482 2023-04-27 16:55:15,810 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/286cf04964004cdfb476199fa40a3fee as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/286cf04964004cdfb476199fa40a3fee 2023-04-27 16:55:15,814 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/286cf04964004cdfb476199fa40a3fee, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:15,815 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/b28132f0b73148bca224f0a96c531482 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/b28132f0b73148bca224f0a96c531482 2023-04-27 16:55:15,819 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/b28132f0b73148bca224f0a96c531482, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:15,820 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 32ms, sequenceid=58, compaction requested=true 2023-04-27 16:55:15,820 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:15,820 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:15,820 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:15,820 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:15,820 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,820 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:15,820 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:15,853 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,853 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:15,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:15,854 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,854 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:15,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:15,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43e2b452[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:15,856 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:15,856 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,954 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:15,954 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:15,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:15,954 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:15,955 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:15,955 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:16,047 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,047 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,047 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:16,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:16,048 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,115 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,115 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,115 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,150 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,151 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,151 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:16,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:16,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:16,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:16,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:16,154 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:16,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:16,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:16,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:16,154 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:16,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,165 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,165 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:16,166 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:16,167 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,167 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,167 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,174 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,174 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,174 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,174 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,175 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:16,175 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,175 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,174 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:16,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,175 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,176 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,275 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:16,287 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:16,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:16,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-27 16:55:16,289 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-27 16:55:16,289 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:16,289 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-27 16:55:16,289 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-27 16:55:16,289 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,289 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,301 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,301 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,301 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,302 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:16,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:16,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:16,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-27 16:55:16,303 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:16,303 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. started... 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:16,303 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:16,303 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:16,311 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/74570c06551e4e978a4b2680a6c749b4 2023-04-27 16:55:16,311 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,322 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,322 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:16,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:16,324 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/6d6c3301c2344f328388efd903033942 2023-04-27 16:55:16,328 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/74570c06551e4e978a4b2680a6c749b4 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/74570c06551e4e978a4b2680a6c749b4 2023-04-27 16:55:16,332 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/74570c06551e4e978a4b2680a6c749b4, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:16,333 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/6d6c3301c2344f328388efd903033942 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/6d6c3301c2344f328388efd903033942 2023-04-27 16:55:16,337 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/6d6c3301c2344f328388efd903033942, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:16,337 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 34ms, sequenceid=71, compaction requested=true 2023-04-27 16:55:16,338 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:16,338 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-27 16:55:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-27 16:55:16,350 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,350 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:16,351 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,351 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-27 16:55:16,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@780b1902[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:16,352 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-27 16:55:16,353 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:16,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,364 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:16,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,364 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:16,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-27 16:55:16,365 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,375 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,375 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:16,385 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-27 16:55:16,385 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,385 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-27 16:55:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-27 16:55:16,388 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-27 16:55:16,388 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:16,388 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:16,388 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-27 16:55:16,388 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:16,395 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,395 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,395 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,395 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,396 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,396 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,398 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,398 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:16,398 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,406 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:16,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-27 16:55:16,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,406 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-27 16:55:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:16,407 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:16,408 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,408 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:16,408 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,408 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-27 16:55:16,408 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-27 16:55:16,507 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:16,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:16,516 INFO [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,517 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1 because midkey is the same as first or last row 2023-04-27 16:55:16,517 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010 because midkey is the same as first or last row 2023-04-27 16:55:16,517 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HRegion(2745): Flushing c38782625824b2e26f5fb73831d5eaef 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:55:16,524 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/b8557b295b2d4e51a991ac02ce1086d3 2023-04-27 16:55:16,536 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/f4fce4d4662d4df190f09d4238a047a2 2023-04-27 16:55:16,539 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf1/b8557b295b2d4e51a991ac02ce1086d3 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/b8557b295b2d4e51a991ac02ce1086d3 2023-04-27 16:55:16,543 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/b8557b295b2d4e51a991ac02ce1086d3, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:16,544 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/.tmp/cf2/f4fce4d4662d4df190f09d4238a047a2 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f4fce4d4662d4df190f09d4238a047a2 2023-04-27 16:55:16,548 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f4fce4d4662d4df190f09d4238a047a2, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:16,549 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c38782625824b2e26f5fb73831d5eaef in 31ms, sequenceid=82, compaction requested=true 2023-04-27 16:55:16,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.HRegion(2446): Flush status journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:16,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1 because midkey is the same as first or last row 2023-04-27 16:55:16,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44953] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010 because midkey is the same as first or last row 2023-04-27 16:55:16,550 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,550 DEBUG [Listener at localhost.localdomain/45365] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-04-27 16:55:16,550 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:16,551 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-04-27 16:55:16,552 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testInvalidSplitKey 2023-04-27 16:55:16,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-27 16:55:16,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614516555"}]},"ts":"1682614516555"} 2023-04-27 16:55:16,556 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-04-27 16:55:16,585 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-04-27 16:55:16,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, UNASSIGN}] 2023-04-27 16:55:16,587 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, UNASSIGN 2023-04-27 16:55:16,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c38782625824b2e26f5fb73831d5eaef, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:16,587 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682614516587"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614516587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614516587"}]},"ts":"1682614516587"} 2023-04-27 16:55:16,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure c38782625824b2e26f5fb73831d5eaef, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:16,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-27 16:55:16,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:16,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing c38782625824b2e26f5fb73831d5eaef, disabling compactions & flushes 2023-04-27 16:55:16,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. after waiting 0 ms 2023-04-27 16:55:16,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:55:16,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:16,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef. 2023-04-27 16:55:16,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for c38782625824b2e26f5fb73831d5eaef: 2023-04-27 16:55:16,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:16,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c38782625824b2e26f5fb73831d5eaef, regionState=CLOSED 2023-04-27 16:55:16,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1682614516748"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614516748"}]},"ts":"1682614516748"} 2023-04-27 16:55:16,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-04-27 16:55:16,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure c38782625824b2e26f5fb73831d5eaef, server=jenkins-hbase12.apache.org,44953,1682614416269 in 160 msec 2023-04-27 16:55:16,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-04-27 16:55:16,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c38782625824b2e26f5fb73831d5eaef, UNASSIGN in 164 msec 2023-04-27 16:55:16,752 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614516752"}]},"ts":"1682614516752"} 2023-04-27 16:55:16,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-04-27 16:55:16,764 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-04-27 16:55:16,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 213 msec 2023-04-27 16:55:16,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-27 16:55:16,858 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-04-27 16:55:16,858 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testInvalidSplitKey 2023-04-27 16:55:16,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-04-27 16:55:16,864 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:16,866 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/recovered.edits] 2023-04-27 16:55:16,869 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/257a38b3ec084840af99f9a102fe86f1 2023-04-27 16:55:16,870 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/286cf04964004cdfb476199fa40a3fee to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/286cf04964004cdfb476199fa40a3fee 2023-04-27 16:55:16,872 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/74570c06551e4e978a4b2680a6c749b4 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/74570c06551e4e978a4b2680a6c749b4 2023-04-27 16:55:16,873 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/a498750b57784238946c983eaf5f1044 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/a498750b57784238946c983eaf5f1044 2023-04-27 16:55:16,874 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/ad2514b75714481dbd232af559ac5212 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/ad2514b75714481dbd232af559ac5212 2023-04-27 16:55:16,875 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/b8557b295b2d4e51a991ac02ce1086d3 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/b8557b295b2d4e51a991ac02ce1086d3 2023-04-27 16:55:16,876 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/e38b95abd6a24bdb83980529801e4857 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf1/e38b95abd6a24bdb83980529801e4857 2023-04-27 16:55:16,877 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/51372a956ce741bdbe44667af9ecc403 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/51372a956ce741bdbe44667af9ecc403 2023-04-27 16:55:16,878 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/5afc18669819444ebad25ceaf5615689 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/5afc18669819444ebad25ceaf5615689 2023-04-27 16:55:16,879 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/6d6c3301c2344f328388efd903033942 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/6d6c3301c2344f328388efd903033942 2023-04-27 16:55:16,880 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/88de9cb62eb242b69207ecfdb14bdb05 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/88de9cb62eb242b69207ecfdb14bdb05 2023-04-27 16:55:16,880 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/b28132f0b73148bca224f0a96c531482 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/b28132f0b73148bca224f0a96c531482 2023-04-27 16:55:16,881 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f4fce4d4662d4df190f09d4238a047a2 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f4fce4d4662d4df190f09d4238a047a2 2023-04-27 16:55:16,882 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/cf2/f57bf8418583423db631f090bc10d010 2023-04-27 16:55:16,883 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef/recovered.edits/85.seqid 2023-04-27 16:55:16,884 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testInvalidSplitKey/c38782625824b2e26f5fb73831d5eaef 2023-04-27 16:55:16,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-04-27 16:55:16,885 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,886 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-04-27 16:55:16,890 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-04-27 16:55:16,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-04-27 16:55:16,891 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614516891"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:16,892 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-27 16:55:16,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c38782625824b2e26f5fb73831d5eaef, NAME => 'testInvalidSplitKey,,1682614514215.c38782625824b2e26f5fb73831d5eaef.', STARTKEY => '', ENDKEY => ''}] 2023-04-27 16:55:16,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-04-27 16:55:16,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614516892"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:16,894 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-04-27 16:55:16,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:16,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 44 msec 2023-04-27 16:55:16,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-04-27 16:55:16,963 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-04-27 16:55:16,975 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=367 (was 346) - Thread LEAK? -, OpenFileDescriptor=740 (was 654) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 215), ProcessCount=171 (was 171), AvailableMemoryMB=11709 (was 11714) 2023-04-27 16:55:16,987 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=367, OpenFileDescriptor=740, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=171, AvailableMemoryMB=11709 2023-04-27 16:55:16,987 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:16,996 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:55:16,996 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:55:16,997 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:16,999 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:55:16,999 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:55:17,001 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,001 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e empty. 2023-04-27 16:55:17,002 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,002 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-04-27 16:55:17,016 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-04-27 16:55:17,017 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 66eafea8e469b1465a6da31506fe876e, NAME => 'testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 66eafea8e469b1465a6da31506fe876e, disabling compactions & flushes 2023-04-27 16:55:17,026 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. after waiting 0 ms 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,026 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,026 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:17,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:55:17,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614517028"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614517028"}]},"ts":"1682614517028"} 2023-04-27 16:55:17,030 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:55:17,030 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:55:17,031 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614517030"}]},"ts":"1682614517030"} 2023-04-27 16:55:17,032 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-04-27 16:55:17,048 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:55:17,049 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:55:17,050 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:55:17,050 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:55:17,050 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:55:17,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, ASSIGN}] 2023-04-27 16:55:17,051 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, ASSIGN 2023-04-27 16:55:17,051 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:17,202 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:55:17,203 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=66eafea8e469b1465a6da31506fe876e, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,203 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614517203"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614517203"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614517203"}]},"ts":"1682614517203"} 2023-04-27 16:55:17,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:17,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 66eafea8e469b1465a6da31506fe876e, NAME => 'testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:55:17,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:17,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:17,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,363 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,364 DEBUG [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1 2023-04-27 16:55:17,364 DEBUG [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1 2023-04-27 16:55:17,364 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66eafea8e469b1465a6da31506fe876e columnFamilyName cf1 2023-04-27 16:55:17,365 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] regionserver.HStore(310): Store=66eafea8e469b1465a6da31506fe876e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:17,365 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,366 DEBUG [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2 2023-04-27 16:55:17,366 DEBUG [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2 2023-04-27 16:55:17,366 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66eafea8e469b1465a6da31506fe876e columnFamilyName cf2 2023-04-27 16:55:17,367 INFO [StoreOpener-66eafea8e469b1465a6da31506fe876e-1] regionserver.HStore(310): Store=66eafea8e469b1465a6da31506fe876e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:17,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:17,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:17,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:55:17,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 66eafea8e469b1465a6da31506fe876e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65376908, jitterRate=-0.02580815553665161}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:17,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:17,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e., pid=111, masterSystemTime=1682614517356 2023-04-27 16:55:17,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,378 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=66eafea8e469b1465a6da31506fe876e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,378 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614517378"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614517378"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614517378"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614517378"}]},"ts":"1682614517378"} 2023-04-27 16:55:17,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-04-27 16:55:17,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 in 174 msec 2023-04-27 16:55:17,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-04-27 16:55:17,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, ASSIGN in 331 msec 2023-04-27 16:55:17,383 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:55:17,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614517383"}]},"ts":"1682614517383"} 2023-04-27 16:55:17,384 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-04-27 16:55:17,428 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:55:17,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 431 msec 2023-04-27 16:55:17,505 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:17,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:17,506 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,507 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,507 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:17,507 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:17,507 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,507 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,519 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,519 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,519 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,520 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,520 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:17,521 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:17,521 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,522 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:17,523 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:55:17,532 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/2d3225c564e24fa297132cde588de696 2023-04-27 16:55:17,532 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,543 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,543 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,543 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,543 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,543 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,543 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/870465df835c49a09ad882e89871535c 2023-04-27 16:55:17,547 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/2d3225c564e24fa297132cde588de696 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 2023-04-27 16:55:17,551 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:17,551 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/870465df835c49a09ad882e89871535c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c 2023-04-27 16:55:17,555 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:17,556 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 33ms, sequenceid=6, compaction requested=false 2023-04-27 16:55:17,556 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,556 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:17,557 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,557 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:17,557 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:17,557 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,557 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,557 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,564 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:17,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@459dd733[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:17,566 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:17,566 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,575 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:17,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:17,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:17,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,595 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,595 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,595 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,596 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,596 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,596 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,596 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:17,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:17,598 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:17,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:17,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:17,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,598 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,607 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,607 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,607 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,617 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,617 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,617 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:17,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,618 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,718 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:17,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:17,730 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:17,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:17,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:17,731 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:17,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:17,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,731 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,743 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,743 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,743 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,743 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:17,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:17,745 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:17,745 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,745 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:17,746 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:17,754 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/5448f5c11fc64ecbac5e60014e68193d 2023-04-27 16:55:17,764 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/86d16757ab414255911db2aa43964dad 2023-04-27 16:55:17,768 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/5448f5c11fc64ecbac5e60014e68193d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d 2023-04-27 16:55:17,772 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:17,772 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/86d16757ab414255911db2aa43964dad as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad 2023-04-27 16:55:17,776 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:17,777 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 31ms, sequenceid=19, compaction requested=false 2023-04-27 16:55:17,777 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:17,777 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,777 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:17,777 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:17,777 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,777 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,777 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,781 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,781 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,803 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,803 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,803 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,803 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,803 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,803 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,804 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,804 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@171bacd3[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:17,805 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:17,805 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:17,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:17,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,838 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,838 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,838 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,848 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:17,848 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,848 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:17,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:17,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:17,852 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:17,852 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:17,852 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:17,852 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,852 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:17,859 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,859 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:17,860 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,861 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,861 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:17,861 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,869 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,870 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:17,869 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:17,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:17,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:17,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:17,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,970 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:17,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:17,981 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:17,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:17,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:17,983 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:17,983 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,983 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:17,983 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:17,984 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,984 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,996 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:17,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:17,996 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,997 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:17,998 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:17,998 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:17,998 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:17,998 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:17,998 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:17,998 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:17,998 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:18,005 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/60f7d11c8b664709ade43c4edb0c08f5 2023-04-27 16:55:18,006 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,016 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,016 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/fe12dae3997d4159b5627c472c9440c2 2023-04-27 16:55:18,016 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,017 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,017 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,017 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,020 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/60f7d11c8b664709ade43c4edb0c08f5 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 2023-04-27 16:55:18,025 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:18,025 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/fe12dae3997d4159b5627c472c9440c2 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 2023-04-27 16:55:18,029 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:18,030 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 32ms, sequenceid=32, compaction requested=true 2023-04-27 16:55:18,030 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:18,030 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,030 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:18,030 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:18,030 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,030 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,030 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,038 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,038 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:18,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@410dca46[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:18,040 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:18,041 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,048 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,049 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:18,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:18,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:18,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,059 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,098 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,098 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:18,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:18,101 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:18,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:18,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:18,102 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,102 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,122 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,123 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,123 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,123 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,125 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,125 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,125 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,132 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,132 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,132 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,132 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,133 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,132 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,133 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,133 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:18,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:18,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,133 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,134 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,134 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,134 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,234 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:18,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:18,246 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:18,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:18,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:18,247 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:18,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:18,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,247 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,259 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,259 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,259 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,259 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,260 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:18,261 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:18,261 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,261 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:18,261 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:18,268 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/3f7217c66f774ecd8797bd13c6204664 2023-04-27 16:55:18,269 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,278 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/a43bf24f73c044a6b7268abff7745080 2023-04-27 16:55:18,280 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,280 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,280 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,280 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,280 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,281 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/3f7217c66f774ecd8797bd13c6204664 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 2023-04-27 16:55:18,285 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:18,285 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/a43bf24f73c044a6b7268abff7745080 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 2023-04-27 16:55:18,289 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:18,290 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 29ms, sequenceid=45, compaction requested=true 2023-04-27 16:55:18,290 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:18,290 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:18,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:18,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,301 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,301 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:18,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@616e086c[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:18,303 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:18,303 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,311 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,311 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,311 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,312 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:18,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:18,312 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:18,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,332 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,332 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,333 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,333 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:18,336 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:18,336 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:18,336 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:18,336 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,336 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,344 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,344 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,344 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,345 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,346 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,346 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,354 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,354 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,354 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:18,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:18,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,354 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,355 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,355 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,355 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,355 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,355 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,455 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:18,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:18,467 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:18,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:18,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:18,468 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,468 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,468 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:18,468 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:18,469 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,469 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,480 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,480 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,480 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,480 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,480 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:18,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:18,482 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:18,482 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,482 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:18,482 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:18,489 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/8f4b48498a9546aaa41f7089fcafdd2e 2023-04-27 16:55:18,499 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/53fc57e9384b4e54b0f720e7bcf1d155 2023-04-27 16:55:18,503 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/8f4b48498a9546aaa41f7089fcafdd2e as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e 2023-04-27 16:55:18,507 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:18,507 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/53fc57e9384b4e54b0f720e7bcf1d155 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 2023-04-27 16:55:18,511 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:18,512 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 30ms, sequenceid=58, compaction requested=true 2023-04-27 16:55:18,512 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:18,512 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,512 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:18,512 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:18,512 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,513 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,513 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,520 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,540 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,540 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,540 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,550 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,550 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,550 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,550 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,551 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,551 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:18,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a617f3d[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:18,553 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:18,553 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,564 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:18,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:18,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,565 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:18,574 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,575 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,585 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:18,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:18,588 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:18,588 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:18,588 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:18,588 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,588 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:18,595 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,595 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,596 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,596 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,597 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,598 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,606 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,607 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,607 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,708 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:18,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:18,719 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:18,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:18,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-27 16:55:18,721 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,721 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,721 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-27 16:55:18,721 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-27 16:55:18,722 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,722 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,732 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,732 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,732 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,732 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,733 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,733 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:18,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:18,735 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,734 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:18,735 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. started... 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,735 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,735 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:18,743 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/b13eb4026a7b417c9e0816dd5c119f6b 2023-04-27 16:55:18,743 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,753 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/d57b66fdab0c47db85152160688e8b4d 2023-04-27 16:55:18,753 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,753 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,753 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,754 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,754 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,757 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/b13eb4026a7b417c9e0816dd5c119f6b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b 2023-04-27 16:55:18,761 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:18,762 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/d57b66fdab0c47db85152160688e8b4d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d 2023-04-27 16:55:18,766 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:18,767 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 32ms, sequenceid=71, compaction requested=true 2023-04-27 16:55:18,767 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:18,767 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:18,767 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:18,767 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:18,767 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,767 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-27 16:55:18,767 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-27 16:55:18,774 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,774 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,775 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,775 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-27 16:55:18,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d62f96[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:18,777 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-27 16:55:18,777 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,785 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:18,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:18,786 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-27 16:55:18,786 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-27 16:55:18,786 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:18,785 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:18,795 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,796 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,796 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,837 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,837 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-27 16:55:18,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:18,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-27 16:55:18,840 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-27 16:55:18,840 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:18,840 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:18,840 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-27 16:55:18,840 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:18,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,859 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,860 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,860 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:18,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,861 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,861 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:18,862 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-27 16:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:18,869 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-27 16:55:18,870 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:18,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,869 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,869 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:18,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:18,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:18,870 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:18,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:18,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,871 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-27 16:55:18,970 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:18,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:19,099 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 6c9ebd47-0a00-4e93-9558-ec52eb9cbf04) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:19,230 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-27 16:55:19,231 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:19,232 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,233 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:19,233 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:19,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:19,482 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:19,482 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:19,483 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:19,483 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:19,483 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:19,485 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-27 16:55:19,486 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:19,486 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:19,486 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:19,486 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:19,487 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,487 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:19,487 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,487 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,488 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,488 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:19,488 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,488 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,489 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,489 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:19,489 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:19,489 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,490 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,490 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,490 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:19,490 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,491 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,491 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,491 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:19,491 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,491 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,492 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,492 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,492 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,493 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:19,494 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,494 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:19,494 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,494 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,494 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,494 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-04-27 16:55:19,495 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:19,495 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:19,496 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614549496 2023-04-27 16:55:19,497 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:19,497 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,499 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:19,500 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:19,500 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:19,500 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614579500 2023-04-27 16:55:19,500 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614639500 2023-04-27 16:55:19,500 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:55:19,500 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:19,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:19,502 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:19,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:19,539 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:55:19,750 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,750 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:19,750 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:19,751 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:19,751 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:19,751 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:19,751 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:19,751 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:19,751 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:19,752 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:19,753 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-27 16:55:19,754 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:19,754 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:19,754 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:19,754 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:19,755 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,756 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:19,756 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,756 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:19,756 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:19,756 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,757 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:19,758 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,759 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,759 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:19,759 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:19,759 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,759 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:19,759 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,759 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,760 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,760 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:55:19,761 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:19,761 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:19,762 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614549761 2023-04-27 16:55:19,763 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:19,763 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,765 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:19,766 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:19,766 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:19,766 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614579766 2023-04-27 16:55:19,766 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614639766 2023-04-27 16:55:19,766 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:19,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:19,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN}] 2023-04-27 16:55:19,769 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:19,769 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:19,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:19,769 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:20,016 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,016 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:20,016 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:20,016 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:20,016 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:20,017 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:20,017 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:20,017 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:20,017 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:20,017 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:20,018 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,019 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,020 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,021 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,021 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:20,021 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:20,021 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,021 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,022 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,022 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:20,022 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,022 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,022 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,023 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:20,023 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,023 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,023 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,024 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,024 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:20,024 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,024 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,025 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:20,026 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,026 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,026 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,026 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-04-27 16:55:20,027 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:20,027 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:20,028 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614550028 2023-04-27 16:55:20,029 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:20,029 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,032 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:20,032 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:20,032 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:20,032 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614580032 2023-04-27 16:55:20,032 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614640032 2023-04-27 16:55:20,032 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:20,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN}] 2023-04-27 16:55:20,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:20,035 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:20,036 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=66eafea8e469b1465a6da31506fe876e, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:20,036 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614520036"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614520036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614520036"}]},"ts":"1682614520036"} 2023-04-27 16:55:20,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:20,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 66eafea8e469b1465a6da31506fe876e, disabling compactions & flushes 2023-04-27 16:55:20,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:20,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:20,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. after waiting 0 ms 2023-04-27 16:55:20,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:20,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 66eafea8e469b1465a6da31506fe876e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:55:20,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 2023-04-27 16:55:20,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/504c0e975981425997f05449e40f45fb 2023-04-27 16:55:20,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 2023-04-27 16:55:20,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:20,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/.tmp/cf2/504c0e975981425997f05449e40f45fb as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb 2023-04-27 16:55:20,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:20,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 66eafea8e469b1465a6da31506fe876e in 31ms, sequenceid=82, compaction requested=true 2023-04-27 16:55:20,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:55:20,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:20,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. 2023-04-27 16:55:20,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 66eafea8e469b1465a6da31506fe876e: 2023-04-27 16:55:20,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,227 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED 2023-04-27 16:55:20,227 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614520227"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614520227"}]},"ts":"1682614520227"} 2023-04-27 16:55:20,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-04-27 16:55:20,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 in 190 msec 2023-04-27 16:55:20,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-04-27 16:55:20,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN in 196 msec 2023-04-27 16:55:20,232 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:20,232 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:20,232 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:20,283 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,283 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:20,283 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:20,283 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:20,284 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:20,284 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:20,284 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:20,284 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:20,284 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:20,285 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:20,286 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,287 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,288 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,289 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,290 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:20,291 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:20,292 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,292 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,292 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,292 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,292 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,292 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:55:20,293 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:20,293 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:20,294 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614550294 2023-04-27 16:55:20,295 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:20,295 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,297 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:20,298 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:20,298 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:20,298 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614580298 2023-04-27 16:55:20,298 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614640298 2023-04-27 16:55:20,298 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:55:20,300 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:20,309 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=66eafea8e469b1465a6da31506fe876e, threads=14 2023-04-27 16:55:20,310 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,310 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,310 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,310 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,310 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,314 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,314 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,314 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,315 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,315 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,315 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,315 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,315 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,318 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,442 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,485 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,487 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,490 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,490 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,493 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,493 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,498 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,499 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,499 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,506 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,508 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,508 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,510 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,510 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 66eafea8e469b1465a6da31506fe876e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:55:20,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:20,536 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:20,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:20,548 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,548 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:20,548 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:20,548 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:20,549 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:20,549 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:20,549 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:20,549 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:20,549 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:20,550 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:20,551 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-27 16:55:20,551 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:20,551 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:20,552 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:20,552 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:20,552 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:20,552 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:20,553 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,553 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:20,553 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,553 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,553 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,554 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,555 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,556 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:20,557 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,557 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,557 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,557 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,558 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:55:20,558 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:20,559 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:20,559 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614550559 2023-04-27 16:55:20,561 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,561 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:20,564 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:20,564 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:20,564 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:20,564 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614580564 2023-04-27 16:55:20,564 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614640564 2023-04-27 16:55:20,564 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:20,571 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=66eafea8e469b1465a6da31506fe876e, threads=14 2023-04-27 16:55:20,572 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,572 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,576 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,576 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,576 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,580 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,581 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,581 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,695 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,700 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,720 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,724 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,732 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,736 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,740 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,748 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,751 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,752 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,760 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,761 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,762 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,762 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e for region: 66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:20,762 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 66eafea8e469b1465a6da31506fe876e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:55:20,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:20,796 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:20,797 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:20,797 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:20,797 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:20,797 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:20,814 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,814 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:20,815 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:20,815 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:20,816 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:20,817 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:20,818 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:20,819 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,820 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:20,821 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:20,821 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,821 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,822 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,822 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,822 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:20,822 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:55:20,823 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:20,823 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:20,823 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614550823 2023-04-27 16:55:20,825 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:20,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,827 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:20,827 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:20,827 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:20,827 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614580827 2023-04-27 16:55:20,827 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614640827 2023-04-27 16:55:20,827 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:55:20,830 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:20,832 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:20,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:20,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:20,832 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:21,078 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:21,078 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:21,079 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:21,080 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,081 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,082 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,083 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:21,084 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,084 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,084 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,084 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,085 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,085 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-27 16:55:21,085 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:21,086 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:21,086 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614551086 2023-04-27 16:55:21,088 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:21,088 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,091 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:21,091 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:21,091 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:21,091 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614581091 2023-04-27 16:55:21,091 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614641091 2023-04-27 16:55:21,091 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:21,091 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:21,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614521093"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1682614521093"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1682614521093"}]},"ts":"1682614521093"} 2023-04-27 16:55:21,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614521093"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614521093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614521093"}]},"ts":"1682614521093"} 2023-04-27 16:55:21,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614521093"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614521093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614521093"}]},"ts":"1682614521093"} 2023-04-27 16:55:21,095 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:21,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:21,095 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:21,341 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,341 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:21,342 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:21,342 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:21,343 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:21,344 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-04-27 16:55:21,344 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:21,344 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:21,345 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:21,345 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:21,345 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:21,345 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:21,345 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,346 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:21,346 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,346 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,346 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,346 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:21,347 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,347 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,347 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,348 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,349 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,349 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:21,349 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,349 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,350 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,350 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,351 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,352 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:21,352 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,352 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:21,352 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:21,352 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:21,352 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,352 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,353 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,353 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,353 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,353 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-04-27 16:55:21,354 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:21,354 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:21,354 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614551354 2023-04-27 16:55:21,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,356 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:21,359 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:21,359 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:21,359 INFO [Listener at localhost.localdomain/45365] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328, table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:21,359 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=dc321411c3484d6b54bd0dba33858c40, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,359 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=0e7c2ec3de615c88fdd400a5f7cba042, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,359 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:21,360 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614581360 2023-04-27 16:55:21,360 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614641360 2023-04-27 16:55:21,360 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-27 16:55:21,360 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1682614521360"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1682614521360"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1682614521360"}]},"ts":"1682614521360"} 2023-04-27 16:55:21,360 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614521360"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614521360"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614521360"}]},"ts":"1682614521360"} 2023-04-27 16:55:21,360 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614521360"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614521360"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614521360"}]},"ts":"1682614521360"} 2023-04-27 16:55:21,362 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:21,364 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:21,364 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:21,364 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:21,610 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:21,610 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:21,611 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:21,611 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:21,611 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:21,612 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-04-27 16:55:21,612 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:21,613 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,614 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,615 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,616 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:21,617 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:21,618 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:21,618 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,618 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,618 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,618 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,618 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,618 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-27 16:55:21,619 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:21,619 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:21,620 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614551620 2023-04-27 16:55:21,621 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:21,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,624 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:21,624 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:21,624 INFO [Listener at localhost.localdomain/45365] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328, table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:21,624 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=dc321411c3484d6b54bd0dba33858c40, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,625 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=0e7c2ec3de615c88fdd400a5f7cba042, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,625 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:21,625 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614581625 2023-04-27 16:55:21,625 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614641625 2023-04-27 16:55:21,625 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:21,625 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:21,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN}] 2023-04-27 16:55:21,627 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:21,628 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:21,628 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:21,875 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,875 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:21,875 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:21,875 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:21,876 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:21,877 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:21,877 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:21,877 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:21,877 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:21,878 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:21,879 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-04-27 16:55:21,879 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:21,879 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:21,879 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,880 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:21,881 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:21,882 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:21,883 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:21,884 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:21,885 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,885 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,885 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,885 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,885 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:21,885 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-27 16:55:21,886 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:21,886 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:21,886 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614551886 2023-04-27 16:55:21,888 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:21,888 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:21,890 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:21,890 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:21,890 INFO [Listener at localhost.localdomain/45365] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328, table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:21,890 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=dc321411c3484d6b54bd0dba33858c40, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,891 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=0e7c2ec3de615c88fdd400a5f7cba042, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-27 16:55:21,891 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:21,891 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614581891 2023-04-27 16:55:21,891 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614641891 2023-04-27 16:55:21,891 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-27 16:55:21,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN}] 2023-04-27 16:55:21,891 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:21,894 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN 2023-04-27 16:55:21,894 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:21,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN 2023-04-27 16:55:21,895 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:22,044 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:55:22,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0e7c2ec3de615c88fdd400a5f7cba042, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522046"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614522046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614522046"}]},"ts":"1682614522046"} 2023-04-27 16:55:22,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:22,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=dc321411c3484d6b54bd0dba33858c40, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522050"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614522050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614522050"}]},"ts":"1682614522050"} 2023-04-27 16:55:22,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:22,101 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid c3e9854d-f6b7-4cf1-bd29-59caf4745df9) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:22,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0e7c2ec3de615c88fdd400a5f7cba042, NAME => 'testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.', STARTKEY => '41', ENDKEY => ''} 2023-04-27 16:55:22,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:22,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:22,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,204 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,204 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1 2023-04-27 16:55:22,204 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1 2023-04-27 16:55:22,205 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e7c2ec3de615c88fdd400a5f7cba042 columnFamilyName cf1 2023-04-27 16:55:22,218 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-top 2023-04-27 16:55:22,221 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-top 2023-04-27 16:55:22,225 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-top 2023-04-27 16:55:22,228 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-top 2023-04-27 16:55:22,231 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-top 2023-04-27 16:55:22,235 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-top 2023-04-27 16:55:22,238 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-top 2023-04-27 16:55:22,238 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(310): Store=0e7c2ec3de615c88fdd400a5f7cba042/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:22,238 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,239 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2 2023-04-27 16:55:22,239 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2 2023-04-27 16:55:22,239 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e7c2ec3de615c88fdd400a5f7cba042 columnFamilyName cf2 2023-04-27 16:55:22,250 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-top 2023-04-27 16:55:22,253 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-top 2023-04-27 16:55:22,257 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-top 2023-04-27 16:55:22,260 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-top 2023-04-27 16:55:22,263 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-top 2023-04-27 16:55:22,266 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-top 2023-04-27 16:55:22,270 DEBUG [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-top 2023-04-27 16:55:22,270 INFO [StoreOpener-0e7c2ec3de615c88fdd400a5f7cba042-1] regionserver.HStore(310): Store=0e7c2ec3de615c88fdd400a5f7cba042/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:22,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:22,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 0e7c2ec3de615c88fdd400a5f7cba042; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68132665, jitterRate=0.015255823731422424}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:22,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 0e7c2ec3de615c88fdd400a5f7cba042: 2023-04-27 16:55:22,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042., pid=117, masterSystemTime=1682614522199 2023-04-27 16:55:22,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:22,277 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:22,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:22,277 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,277 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): 0e7c2ec3de615c88fdd400a5f7cba042/cf1 is initiating minor compaction (all files) 2023-04-27 16:55:22,277 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1912): 0e7c2ec3de615c88fdd400a5f7cba042/cf2 is initiating minor compaction (all files) 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0e7c2ec3de615c88fdd400a5f7cba042/cf1 in testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0e7c2ec3de615c88fdd400a5f7cba042/cf2 in testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/.tmp, totalSize=34.9 K 2023-04-27 16:55:22,277 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/.tmp, totalSize=34.9 K 2023-04-27 16:55:22,278 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614517500 2023-04-27 16:55:22,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:22,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dc321411c3484d6b54bd0dba33858c40, NAME => 'testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.', STARTKEY => '', ENDKEY => '41'} 2023-04-27 16:55:22,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:22,279 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0e7c2ec3de615c88fdd400a5f7cba042, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:22,279 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522279"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614522279"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614522279"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614522279"}]},"ts":"1682614522279"} 2023-04-27 16:55:22,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,279 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614517500 2023-04-27 16:55:22,280 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614517719 2023-04-27 16:55:22,280 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614517972 2023-04-27 16:55:22,281 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614517719 2023-04-27 16:55:22,281 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614518236 2023-04-27 16:55:22,281 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614517972 2023-04-27 16:55:22,281 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614518457 2023-04-27 16:55:22,281 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614518236 2023-04-27 16:55:22,282 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614518457 2023-04-27 16:55:22,282 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614518709 2023-04-27 16:55:22,282 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,282 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614518709 2023-04-27 16:55:22,283 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614518972 2023-04-27 16:55:22,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-04-27 16:55:22,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328 in 233 msec 2023-04-27 16:55:22,283 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614518972 2023-04-27 16:55:22,283 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1 2023-04-27 16:55:22,283 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1 2023-04-27 16:55:22,284 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc321411c3484d6b54bd0dba33858c40 columnFamilyName cf1 2023-04-27 16:55:22,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN in 392 msec 2023-04-27 16:55:22,296 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] throttle.PressureAwareThroughputController(145): 0e7c2ec3de615c88fdd400a5f7cba042#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:22,300 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0e7c2ec3de615c88fdd400a5f7cba042#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:22,305 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-bottom 2023-04-27 16:55:22,307 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-bottom 2023-04-27 16:55:22,314 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-bottom 2023-04-27 16:55:22,318 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-bottom 2023-04-27 16:55:22,324 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-bottom 2023-04-27 16:55:22,331 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-bottom 2023-04-27 16:55:22,335 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/.tmp/cf1/0f189175a9d04597b30199235a481b80 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/0f189175a9d04597b30199235a481b80 2023-04-27 16:55:22,336 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-bottom 2023-04-27 16:55:22,337 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(310): Store=dc321411c3484d6b54bd0dba33858c40/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:22,337 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,338 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2 2023-04-27 16:55:22,338 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2 2023-04-27 16:55:22,338 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc321411c3484d6b54bd0dba33858c40 columnFamilyName cf2 2023-04-27 16:55:22,339 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e7c2ec3de615c88fdd400a5f7cba042/cf1 of 0e7c2ec3de615c88fdd400a5f7cba042 into 0f189175a9d04597b30199235a481b80(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:22,339 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0e7c2ec3de615c88fdd400a5f7cba042: 2023-04-27 16:55:22,339 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042., storeName=0e7c2ec3de615c88fdd400a5f7cba042/cf1, priority=9, startTime=1682614522276; duration=0sec 2023-04-27 16:55:22,340 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:22,348 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-bottom 2023-04-27 16:55:22,351 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-bottom 2023-04-27 16:55:22,354 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-bottom 2023-04-27 16:55:22,358 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-bottom 2023-04-27 16:55:22,361 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-bottom 2023-04-27 16:55:22,364 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-bottom 2023-04-27 16:55:22,367 DEBUG [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-bottom 2023-04-27 16:55:22,367 INFO [StoreOpener-dc321411c3484d6b54bd0dba33858c40-1] regionserver.HStore(310): Store=dc321411c3484d6b54bd0dba33858c40/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:22,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:22,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:22,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened dc321411c3484d6b54bd0dba33858c40; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74123305, jitterRate=0.10452331602573395}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:22,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for dc321411c3484d6b54bd0dba33858c40: 2023-04-27 16:55:22,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40., pid=118, masterSystemTime=1682614522199 2023-04-27 16:55:22,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:22,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:22,372 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:22,373 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,373 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): dc321411c3484d6b54bd0dba33858c40/cf1 is initiating minor compaction (all files) 2023-04-27 16:55:22,373 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dc321411c3484d6b54bd0dba33858c40/cf1 in testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,373 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/.tmp, totalSize=34.9 K 2023-04-27 16:55:22,374 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614517500 2023-04-27 16:55:22,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,374 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614517719 2023-04-27 16:55:22,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=dc321411c3484d6b54bd0dba33858c40, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522374"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614522374"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614522374"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614522374"}]},"ts":"1682614522374"} 2023-04-27 16:55:22,374 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614517972 2023-04-27 16:55:22,375 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614518236 2023-04-27 16:55:22,375 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614518457 2023-04-27 16:55:22,376 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614518709 2023-04-27 16:55:22,376 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614518972 2023-04-27 16:55:22,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-04-27 16:55:22,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328 in 324 msec 2023-04-27 16:55:22,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-04-27 16:55:22,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN in 486 msec 2023-04-27 16:55:22,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-27 16:55:22,378 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:22,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-27 16:55:22,383 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): dc321411c3484d6b54bd0dba33858c40#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:22,391 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,391 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-27 16:55:22,391 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:22,392 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:22,393 ERROR [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-27 16:55:22,393 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/.tmp/cf1/9fdf89c4cf8e4f0594177ba767b3b76b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/9fdf89c4cf8e4f0594177ba767b3b76b 2023-04-27 16:55:22,393 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-27 16:55:22,393 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-27 16:55:22,393 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-27 16:55:22,393 INFO [Listener at localhost.localdomain/45365] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-27 16:55:22,394 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-27 16:55:22,395 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-04-27 16:55:22,395 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-04-27 16:55:22,396 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-27 16:55:22,397 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-27 16:55:22,397 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-27 16:55:22,397 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-27 16:55:22,397 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:22,397 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dc321411c3484d6b54bd0dba33858c40/cf1 of dc321411c3484d6b54bd0dba33858c40 into 9fdf89c4cf8e4f0594177ba767b3b76b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:22,397 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dc321411c3484d6b54bd0dba33858c40: 2023-04-27 16:55:22,397 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=6ce00783d83d84715d1e04b95abdb46b, daughterA=7885756366b98522751324208da7edc7, daughterB=7c40e5afe141126a7424fd4ac3bbbe94 2023-04-27 16:55:22,398 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40., storeName=dc321411c3484d6b54bd0dba33858c40/cf1, priority=9, startTime=1682614522372; duration=0sec 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:22,398 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-27 16:55:22,398 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:22,398 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:22,398 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): dc321411c3484d6b54bd0dba33858c40/cf2 is initiating minor compaction (all files) 2023-04-27 16:55:22,398 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dc321411c3484d6b54bd0dba33858c40/cf2 in testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=985ef773fc4ca76ee5d569ef4a9b03cc, daughterA=537cccd40f1fb32014621fe16dc91849, daughterB=0c87014d6264f49d86caaf77094e33e7 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:22,398 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-27 16:55:22,399 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/.tmp, totalSize=34.9 K 2023-04-27 16:55:22,399 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:22,399 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=816f71b9fefe9bb30d3dc5b1086f6f2c, daughterA=b916b9c9bd3e02ea62ae1ce105b3e45a, daughterB=c7106e59e45afb6f541ec61fd2406ddc 2023-04-27 16:55:22,399 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=816f71b9fefe9bb30d3dc5b1086f6f2c, ASSIGN 2023-04-27 16:55:22,399 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:22,399 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614517500 2023-04-27 16:55:22,399 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-27 16:55:22,400 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:22,400 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=4da958a3a9419b0e6e2a5a7fefc50972, daughterA=bd98018c89359efe591534fa1b58d354, daughterB=1b60f0ca3dad9cb4e6fbe55481d58af3 2023-04-27 16:55:22,400 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614517719 2023-04-27 16:55:22,400 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:22,400 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-27 16:55:22,400 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:22,400 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614517972 2023-04-27 16:55:22,401 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ca3d7a0582cb383ce3d284ca8b366c1d, daughterA=758574340f27bbce277a4f2e49747b8e, daughterB=a8a9f5a06f80540e69723d7fa4cd82c8 2023-04-27 16:55:22,401 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:22,401 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-27 16:55:22,401 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614518236 2023-04-27 16:55:22,401 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:22,401 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:22,401 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614518457 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,46011,1682614416374, table=testRollbackForSplitTableRegionWithReplica, region=45bbccc83e9d10da908026afa9931611 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=36ff433ee5b46c5cccef4e1cd0657044, daughterA=5eddd010940c3b7dab205cfe1a311fb8, daughterB=c42021d0058ce12c7842461c449cf609 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-27 16:55:22,402 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614518709 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:22,402 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-27 16:55:22,402 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614518972 2023-04-27 16:55:22,403 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN 2023-04-27 16:55:22,404 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 held the lock before restarting, call acquireLock to restore it. 2023-04-27 16:55:22,405 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,405 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,405 INFO [Listener at localhost.localdomain/45365] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 66eafea8e469b1465a6da31506fe876e, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,405 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,406 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,406 DEBUG [Listener at localhost.localdomain/45365] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328 didn't hold the lock before restarting, skip acquiring lock. 2023-04-27 16:55:22,406 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-04-27 16:55:22,406 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-27 16:55:22,407 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46011,1682614416374, table=hbase:meta, region=1588230740 2023-04-27 16:55:22,407 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1682614552407 2023-04-27 16:55:22,409 DEBUG [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1517): Joining cluster... 2023-04-27 16:55:22,409 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:22,411 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=a181d38dc7ff4539b60029e08bf407bf, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,46011,1682614416374, regionLocation=jenkins-hbase12.apache.org,46011,1682614416374, openSeqNum=2 2023-04-27 16:55:22,411 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=66eafea8e469b1465a6da31506fe876e, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=2 2023-04-27 16:55:22,411 INFO [Listener at localhost.localdomain/45365] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41143,1682614416328, table=testRecoveryAndDoubleExecution, region=66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:22,412 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=dc321411c3484d6b54bd0dba33858c40, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=86 2023-04-27 16:55:22,412 DEBUG [Listener at localhost.localdomain/45365] assignment.RegionStateStore(147): Load hbase:meta entry region=0e7c2ec3de615c88fdd400a5f7cba042, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41143,1682614416328, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328, openSeqNum=86 2023-04-27 16:55:22,412 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-27 16:55:22,412 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1682614582412 2023-04-27 16:55:22,412 INFO [Listener at localhost.localdomain/45365] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1682614642412 2023-04-27 16:55:22,412 INFO [Listener at localhost.localdomain/45365] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-27 16:55:22,412 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-27 16:55:22,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=66eafea8e469b1465a6da31506fe876e, daughterA=dc321411c3484d6b54bd0dba33858c40, daughterB=0e7c2ec3de615c88fdd400a5f7cba042 in 3.1810 sec 2023-04-27 16:55:22,416 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): dc321411c3484d6b54bd0dba33858c40#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:22,432 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/.tmp/cf2/25f12ce768974ce2b028d688df5e0563 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/25f12ce768974ce2b028d688df5e0563 2023-04-27 16:55:22,436 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dc321411c3484d6b54bd0dba33858c40/cf2 of dc321411c3484d6b54bd0dba33858c40 into 25f12ce768974ce2b028d688df5e0563(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:22,436 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dc321411c3484d6b54bd0dba33858c40: 2023-04-27 16:55:22,437 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40., storeName=dc321411c3484d6b54bd0dba33858c40/cf2, priority=9, startTime=1682614522372; duration=0sec 2023-04-27 16:55:22,437 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:22,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41143] ipc.CallRunner(144): callId: 902 service: ClientService methodName: Scan size: 154 connection: 136.243.104.168:49980 deadline: 1682614582663, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e. is not online on jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,719 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/.tmp/cf2/e861513e7e7349bba9564a734fd84ac3 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/e861513e7e7349bba9564a734fd84ac3 2023-04-27 16:55:22,724 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e7c2ec3de615c88fdd400a5f7cba042/cf2 of 0e7c2ec3de615c88fdd400a5f7cba042 into e861513e7e7349bba9564a734fd84ac3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:22,724 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0e7c2ec3de615c88fdd400a5f7cba042: 2023-04-27 16:55:22,724 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042., storeName=0e7c2ec3de615c88fdd400a5f7cba042/cf2, priority=9, startTime=1682614522277; duration=0sec 2023-04-27 16:55:22,724 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:22,774 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:55:22,782 INFO [Listener at localhost.localdomain/45365] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-27 16:55:22,783 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:55:22,792 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:22,794 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-04-27 16:55:22,794 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testRecoveryAndDoubleExecution 2023-04-27 16:55:22,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:22,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-27 16:55:22,797 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614522797"}]},"ts":"1682614522797"} 2023-04-27 16:55:22,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-04-27 16:55:22,869 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-04-27 16:55:22,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, UNASSIGN}] 2023-04-27 16:55:22,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, UNASSIGN 2023-04-27 16:55:22,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0e7c2ec3de615c88fdd400a5f7cba042, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522871"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614522871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614522871"}]},"ts":"1682614522871"} 2023-04-27 16:55:22,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:22,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, UNASSIGN 2023-04-27 16:55:22,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=dc321411c3484d6b54bd0dba33858c40, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:22,875 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614522875"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614522875"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614522875"}]},"ts":"1682614522875"} 2023-04-27 16:55:22,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:22,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-27 16:55:23,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:23,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 0e7c2ec3de615c88fdd400a5f7cba042, disabling compactions & flushes 2023-04-27 16:55:23,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:23,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:23,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. after waiting 0 ms 2023-04-27 16:55:23,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:23,025 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-top] to archive 2023-04-27 16:55:23,026 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:23,027 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,028 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,029 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,030 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,032 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,033 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,034 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,036 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-top] to archive 2023-04-27 16:55:23,036 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:23,038 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,040 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,041 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,042 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,043 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,044 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,045 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:55:23,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:23,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042. 2023-04-27 16:55:23,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 0e7c2ec3de615c88fdd400a5f7cba042: 2023-04-27 16:55:23,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:23,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:23,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing dc321411c3484d6b54bd0dba33858c40, disabling compactions & flushes 2023-04-27 16:55:23,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:23,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:23,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. after waiting 0 ms 2023-04-27 16:55:23,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0e7c2ec3de615c88fdd400a5f7cba042, regionState=CLOSED 2023-04-27 16:55:23,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:23,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614523052"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614523052"}]},"ts":"1682614523052"} 2023-04-27 16:55:23,053 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4-bottom] to archive 2023-04-27 16:55:23,054 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:23,055 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,056 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-04-27 16:55:23,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0e7c2ec3de615c88fdd400a5f7cba042, server=jenkins-hbase12.apache.org,41143,1682614416328 in 181 msec 2023-04-27 16:55:23,057 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e7c2ec3de615c88fdd400a5f7cba042, UNASSIGN in 187 msec 2023-04-27 16:55:23,058 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,059 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,060 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,060 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,062 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb-bottom] to archive 2023-04-27 16:55:23,063 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:23,064 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,065 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,066 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,067 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,068 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,069 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,070 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:55:23,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:23,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40. 2023-04-27 16:55:23,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for dc321411c3484d6b54bd0dba33858c40: 2023-04-27 16:55:23,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:23,076 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=dc321411c3484d6b54bd0dba33858c40, regionState=CLOSED 2023-04-27 16:55:23,076 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1682614523076"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614523076"}]},"ts":"1682614523076"} 2023-04-27 16:55:23,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-04-27 16:55:23,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure dc321411c3484d6b54bd0dba33858c40, server=jenkins-hbase12.apache.org,41143,1682614416328 in 202 msec 2023-04-27 16:55:23,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-04-27 16:55:23,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dc321411c3484d6b54bd0dba33858c40, UNASSIGN in 209 msec 2023-04-27 16:55:23,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614523080"}]},"ts":"1682614523080"} 2023-04-27 16:55:23,081 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-04-27 16:55:23,090 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-04-27 16:55:23,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 297 msec 2023-04-27 16:55:23,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-27 16:55:23,100 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-04-27 16:55:23,100 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testRecoveryAndDoubleExecution 2023-04-27 16:55:23,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,103 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,103 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-04-27 16:55:23,108 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,108 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:23,111 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:23,112 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/recovered.edits] 2023-04-27 16:55:23,112 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/recovered.edits] 2023-04-27 16:55:23,113 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/recovered.edits] 2023-04-27 16:55:23,115 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/9fdf89c4cf8e4f0594177ba767b3b76b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf1/9fdf89c4cf8e4f0594177ba767b3b76b 2023-04-27 16:55:23,116 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/2d3225c564e24fa297132cde588de696 2023-04-27 16:55:23,116 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/.regioninfo to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/.regioninfo 2023-04-27 16:55:23,117 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/3f7217c66f774ecd8797bd13c6204664 2023-04-27 16:55:23,117 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/25f12ce768974ce2b028d688df5e0563 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/cf2/25f12ce768974ce2b028d688df5e0563 2023-04-27 16:55:23,118 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,119 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4 2023-04-27 16:55:23,120 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,120 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/5448f5c11fc64ecbac5e60014e68193d 2023-04-27 16:55:23,124 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/.regioninfo to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/.regioninfo 2023-04-27 16:55:23,125 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,125 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/60f7d11c8b664709ade43c4edb0c08f5 2023-04-27 16:55:23,126 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,126 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/8f4b48498a9546aaa41f7089fcafdd2e 2023-04-27 16:55:23,127 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,127 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/2d3225c564e24fa297132cde588de696.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,127 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf1/b13eb4026a7b417c9e0816dd5c119f6b 2023-04-27 16:55:23,128 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,128 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/3f7217c66f774ecd8797bd13c6204664.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,129 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,129 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/47bf1b855faf4bd0a7b6b56e0a2911b4.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,130 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/504c0e975981425997f05449e40f45fb 2023-04-27 16:55:23,130 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/5448f5c11fc64ecbac5e60014e68193d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,131 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/53fc57e9384b4e54b0f720e7bcf1d155 2023-04-27 16:55:23,131 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/60f7d11c8b664709ade43c4edb0c08f5.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,132 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,132 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/86d16757ab414255911db2aa43964dad 2023-04-27 16:55:23,132 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/8f4b48498a9546aaa41f7089fcafdd2e.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,134 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,134 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/870465df835c49a09ad882e89871535c 2023-04-27 16:55:23,135 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf1/b13eb4026a7b417c9e0816dd5c119f6b.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,136 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,136 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/a43bf24f73c044a6b7268abff7745080 2023-04-27 16:55:23,137 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/d57b66fdab0c47db85152160688e8b4d 2023-04-27 16:55:23,137 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,138 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/504c0e975981425997f05449e40f45fb.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,139 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,139 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/cf2/fe12dae3997d4159b5627c472c9440c2 2023-04-27 16:55:23,139 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/53fc57e9384b4e54b0f720e7bcf1d155.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,140 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,140 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/86d16757ab414255911db2aa43964dad.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,142 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e/recovered.edits/85.seqid 2023-04-27 16:55:23,142 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/0e7c2ec3de615c88fdd400a5f7cba042/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,142 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/870465df835c49a09ad882e89871535c.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,143 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,144 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/a43bf24f73c044a6b7268abff7745080.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,144 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/0f189175a9d04597b30199235a481b80 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf1/0f189175a9d04597b30199235a481b80 2023-04-27 16:55:23,145 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/d57b66fdab0c47db85152160688e8b4d.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,146 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/dc321411c3484d6b54bd0dba33858c40/cf2/fe12dae3997d4159b5627c472c9440c2.66eafea8e469b1465a6da31506fe876e 2023-04-27 16:55:23,147 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/e861513e7e7349bba9564a734fd84ac3 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/cf2/e861513e7e7349bba9564a734fd84ac3 2023-04-27 16:55:23,148 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40/recovered.edits/90.seqid 2023-04-27 16:55:23,149 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/dc321411c3484d6b54bd0dba33858c40 2023-04-27 16:55:23,149 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042/recovered.edits/90.seqid 2023-04-27 16:55:23,149 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testRecoveryAndDoubleExecution/0e7c2ec3de615c88fdd400a5f7cba042 2023-04-27 16:55:23,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-04-27 16:55:23,152 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,153 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-04-27 16:55:23,155 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-04-27 16:55:23,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-04-27 16:55:23,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614523156"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:23,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614523156"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:23,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614523156"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:23,158 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:55:23,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 66eafea8e469b1465a6da31506fe876e, NAME => 'testRecoveryAndDoubleExecution,,1682614516997.66eafea8e469b1465a6da31506fe876e.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => dc321411c3484d6b54bd0dba33858c40, NAME => 'testRecoveryAndDoubleExecution,,1682614519231.dc321411c3484d6b54bd0dba33858c40.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0e7c2ec3de615c88fdd400a5f7cba042, NAME => 'testRecoveryAndDoubleExecution,41,1682614519231.0e7c2ec3de615c88fdd400a5f7cba042.', STARTKEY => '41', ENDKEY => ''}] 2023-04-27 16:55:23,158 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-04-27 16:55:23,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614523158"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:23,160 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-04-27 16:55:23,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-27 16:55:23,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 72 msec 2023-04-27 16:55:23,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-04-27 16:55:23,205 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-04-27 16:55:23,220 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 367) - Thread LEAK? -, OpenFileDescriptor=842 (was 740) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 215), ProcessCount=171 (was 171), AvailableMemoryMB=11650 (was 11709) 2023-04-27 16:55:23,234 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=197, ProcessCount=171, AvailableMemoryMB=11649 2023-04-27 16:55:23,234 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:23,245 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:55:23,245 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:55:23,247 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:23,248 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:55:23,249 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:55:23,251 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:23,251 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 empty. 2023-04-27 16:55:23,251 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:23,251 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-04-27 16:55:23,661 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-04-27 16:55:23,662 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 38bab30e6c18227e87f7eed5826a9e64, NAME => 'testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 38bab30e6c18227e87f7eed5826a9e64, disabling compactions & flushes 2023-04-27 16:55:23,669 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. after waiting 0 ms 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:23,669 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:23,671 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:55:23,671 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614523671"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614523671"}]},"ts":"1682614523671"} 2023-04-27 16:55:23,672 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:55:23,673 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:55:23,673 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614523673"}]},"ts":"1682614523673"} 2023-04-27 16:55:23,674 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-04-27 16:55:23,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:55:23,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:55:23,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:55:23,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:55:23,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:55:23,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, ASSIGN}] 2023-04-27 16:55:23,735 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, ASSIGN 2023-04-27 16:55:23,735 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=false 2023-04-27 16:55:23,885 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:55:23,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=38bab30e6c18227e87f7eed5826a9e64, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:23,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614523887"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614523887"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614523887"}]},"ts":"1682614523887"} 2023-04-27 16:55:23,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 38bab30e6c18227e87f7eed5826a9e64, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:24,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 38bab30e6c18227e87f7eed5826a9e64, NAME => 'testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:55:24,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:24,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:24,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,044 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,045 DEBUG [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1 2023-04-27 16:55:24,045 DEBUG [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1 2023-04-27 16:55:24,045 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38bab30e6c18227e87f7eed5826a9e64 columnFamilyName cf1 2023-04-27 16:55:24,046 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] regionserver.HStore(310): Store=38bab30e6c18227e87f7eed5826a9e64/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:24,046 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,047 DEBUG [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2 2023-04-27 16:55:24,047 DEBUG [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2 2023-04-27 16:55:24,047 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38bab30e6c18227e87f7eed5826a9e64 columnFamilyName cf2 2023-04-27 16:55:24,048 INFO [StoreOpener-38bab30e6c18227e87f7eed5826a9e64-1] regionserver.HStore(310): Store=38bab30e6c18227e87f7eed5826a9e64/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:24,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:24,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:24,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:55:24,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 38bab30e6c18227e87f7eed5826a9e64; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63960316, jitterRate=-0.04691702127456665}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:24,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:24,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64., pid=127, masterSystemTime=1682614524040 2023-04-27 16:55:24,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,055 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=38bab30e6c18227e87f7eed5826a9e64, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,055 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614524055"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614524055"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614524055"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614524055"}]},"ts":"1682614524055"} 2023-04-27 16:55:24,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-04-27 16:55:24,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 38bab30e6c18227e87f7eed5826a9e64, server=jenkins-hbase12.apache.org,44953,1682614416269 in 169 msec 2023-04-27 16:55:24,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-04-27 16:55:24,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, ASSIGN in 324 msec 2023-04-27 16:55:24,060 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:55:24,060 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614524060"}]},"ts":"1682614524060"} 2023-04-27 16:55:24,061 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-04-27 16:55:24,081 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:55:24,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 835 msec 2023-04-27 16:55:24,255 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:24,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,256 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,256 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,256 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:24,256 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:24,257 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,257 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,269 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,269 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,269 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,269 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,270 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,270 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,271 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:24,272 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:24,272 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,272 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:24,273 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:55:24,279 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,281 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/56b533a1cbca4b40a6d2e3f787d1860b 2023-04-27 16:55:24,290 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,290 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,290 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,290 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,292 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/8bfe3bc49444479eac4fa9d56e3e5380 2023-04-27 16:55:24,295 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/56b533a1cbca4b40a6d2e3f787d1860b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b 2023-04-27 16:55:24,299 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:24,299 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/8bfe3bc49444479eac4fa9d56e3e5380 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380 2023-04-27 16:55:24,302 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:24,303 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 30ms, sequenceid=6, compaction requested=false 2023-04-27 16:55:24,303 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,303 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:24,303 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:24,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:24,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,303 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,311 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,311 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,312 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:24,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fa68a30[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:24,313 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:24,313 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,322 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:24,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,322 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,322 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,322 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:24,322 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,323 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,323 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:24,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,323 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,323 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,323 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:24,332 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,332 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,332 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,332 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,333 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,343 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,343 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,343 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:24,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:24,346 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:24,346 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:24,346 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:24,346 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,346 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,354 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,353 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,354 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,355 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,356 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,356 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,387 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,387 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:24,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,388 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,488 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:24,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:24,500 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:24,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:24,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,501 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,501 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,501 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:24,501 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:24,502 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,502 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,522 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,522 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,522 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:24,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,523 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,523 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:24,524 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:24,524 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,524 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:24,524 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:24,532 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/e11b153c1bf84a24b2fe8cf3f970527b 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,532 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,541 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/a41dc73a313046eebe48c6030865230e 2023-04-27 16:55:24,545 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/e11b153c1bf84a24b2fe8cf3f970527b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b 2023-04-27 16:55:24,549 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:24,549 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/a41dc73a313046eebe48c6030865230e as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e 2023-04-27 16:55:24,553 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:24,553 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 29ms, sequenceid=19, compaction requested=false 2023-04-27 16:55:24,553 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:24,554 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,554 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:24,554 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:24,554 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,554 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,554 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,564 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,564 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,564 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,564 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:24,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e1acac0[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:24,566 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:24,566 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,574 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,575 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,575 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:24,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,575 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:24,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,585 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,595 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,595 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,595 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,606 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,606 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,606 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,606 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:24,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:24,609 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:24,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:24,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:24,609 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,609 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:24,616 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,616 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,616 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,616 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,616 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,617 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,617 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,617 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,618 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,619 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,619 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,619 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,627 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,627 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,627 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,627 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,628 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,728 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:24,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:24,738 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:24,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:24,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,740 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,740 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,740 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:24,740 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:24,740 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,740 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,750 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,750 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,750 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:24,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,752 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,752 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,752 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,755 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:24,756 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,755 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,756 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,756 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,764 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/fb06af5fbe9041f9870c57aebd51490f 2023-04-27 16:55:24,775 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/cf25508341ba4eccb25f6957be766eaa 2023-04-27 16:55:24,779 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/fb06af5fbe9041f9870c57aebd51490f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f 2023-04-27 16:55:24,783 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:24,783 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/cf25508341ba4eccb25f6957be766eaa as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa 2023-04-27 16:55:24,786 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:24,787 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 31ms, sequenceid=32, compaction requested=true 2023-04-27 16:55:24,787 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:24,787 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:24,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:24,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,788 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,790 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,790 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,806 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,806 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:24,809 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:24,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f43d2a9[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:24,809 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:24,817 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:24,817 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:24,818 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,827 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,827 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,837 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:24,838 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,838 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:24,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:24,841 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:24,841 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:24,841 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:24,841 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,841 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,849 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,849 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,850 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:24,851 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,851 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,858 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,858 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,859 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:24,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,859 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:24,858 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:24,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,859 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:24,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,860 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,959 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:24,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:24,971 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:24,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:24,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,973 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:24,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:24,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:24,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,985 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,985 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:24,985 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:24,986 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:24,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:24,987 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,987 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,987 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:24,987 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:24,994 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/de2833a7f72a4328b2ae095205ed07d3 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:24,995 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:25,010 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/c6300601a8c94c058b37924fa37aefb8 2023-04-27 16:55:25,013 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/de2833a7f72a4328b2ae095205ed07d3 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3 2023-04-27 16:55:25,017 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:25,017 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/c6300601a8c94c058b37924fa37aefb8 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8 2023-04-27 16:55:25,021 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:25,022 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 35ms, sequenceid=45, compaction requested=true 2023-04-27 16:55:25,022 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:25,022 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:25,022 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:25,022 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:25,022 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,022 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,022 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,034 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,035 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,035 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:25,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e58e848[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:25,036 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:25,037 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,048 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,048 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,048 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,048 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:25,049 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,049 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:25,058 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,058 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,058 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,069 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:25,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:25,072 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:25,072 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:25,072 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:25,072 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,072 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:25,079 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,079 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,079 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,079 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,080 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,080 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,081 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,082 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,082 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,082 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,102 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 5799fd8f-0a4b-4c91-a1ff-1fe687f21837) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:25,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:25,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:25,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,128 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,128 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:25,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,129 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,130 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,130 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,130 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,229 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:25,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:25,240 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:25,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:25,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,242 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:25,242 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,242 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:25,242 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:25,242 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,242 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,257 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,257 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,258 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,258 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:25,259 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,259 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,259 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:25,260 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,260 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,260 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:25,267 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/709a8a4ea18c48408ab7a652fd1a15d4 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:25,268 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:25,679 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/baf010430fb349cda15664887e1bba8c 2023-04-27 16:55:25,683 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/709a8a4ea18c48408ab7a652fd1a15d4 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4 2023-04-27 16:55:25,687 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:25,687 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/baf010430fb349cda15664887e1bba8c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c 2023-04-27 16:55:25,691 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:25,691 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 431ms, sequenceid=58, compaction requested=true 2023-04-27 16:55:25,691 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:25,692 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:25,692 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:25,692 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:25,692 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,692 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,692 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,726 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:55:25,762 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,763 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,763 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:25,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55b40cb9[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:25,765 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:25,765 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,773 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:25,773 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,773 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:25,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:25,774 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:25,783 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,784 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,784 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,794 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:25,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,794 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:25,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:25,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:25,797 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:25,797 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:25,797 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:25,798 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,798 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:25,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:25,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:25,805 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,805 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,806 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:25,806 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:25,808 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,808 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,808 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:25,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,815 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,815 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:25,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:25,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:25,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,816 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,916 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:25,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:25,926 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:25,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:25,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,928 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:25,928 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,928 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-27 16:55:25,928 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-27 16:55:25,928 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,928 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,939 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,939 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,939 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,939 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:25,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:25,940 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,940 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,941 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,942 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:25,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,942 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. started... 2023-04-27 16:55:25,942 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:25,949 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/06e02cc570c34437b8db95a2b57aa7ba 2023-04-27 16:55:25,962 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/d114e522b1ee41c7a68dc608437c266c 2023-04-27 16:55:25,965 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/06e02cc570c34437b8db95a2b57aa7ba as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba 2023-04-27 16:55:25,968 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:25,969 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/d114e522b1ee41c7a68dc608437c266c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c 2023-04-27 16:55:25,972 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:25,973 INFO [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 31ms, sequenceid=71, compaction requested=true 2023-04-27 16:55:25,973 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:25,973 DEBUG [rs(jenkins-hbase12.apache.org,44953,1682614416269)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:25,973 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:25,973 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:25,973 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:25,973 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-27 16:55:25,973 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-27 16:55:25,982 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,003 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,003 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,003 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:26,015 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,015 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:26,015 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:26,015 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:26,016 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,016 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:26,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:26,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:26,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-27 16:55:26,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d909005[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:26,017 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-27 16:55:26,017 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:26,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,026 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,026 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-27 16:55:26,027 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:26,026 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-27 16:55:26,027 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:26,027 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:26,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:26,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:26,036 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:26,047 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:26,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:26,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:26,047 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-27 16:55:26,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:26,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:26,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:26,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,44953,1682614416269': 2023-04-27 16:55:26,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-27 16:55:26,051 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-27 16:55:26,051 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:26,051 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:26,051 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,051 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,057 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:26,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,058 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:26,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,058 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,060 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:26,060 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,060 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-27 16:55:26,068 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:26,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,068 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:26,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,069 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-27 16:55:26,169 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:26,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:26,177 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,178 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=38bab30e6c18227e87f7eed5826a9e64, daughterA=6871275d65296b087ab904889627a9f0, daughterB=18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,179 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=38bab30e6c18227e87f7eed5826a9e64, daughterA=6871275d65296b087ab904889627a9f0, daughterB=18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,179 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=38bab30e6c18227e87f7eed5826a9e64, daughterA=6871275d65296b087ab904889627a9f0, daughterB=18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,179 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=38bab30e6c18227e87f7eed5826a9e64, daughterA=6871275d65296b087ab904889627a9f0, daughterB=18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, UNASSIGN}] 2023-04-27 16:55:26,182 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, UNASSIGN 2023-04-27 16:55:26,182 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=38bab30e6c18227e87f7eed5826a9e64, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,182 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614526182"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614526182"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614526182"}]},"ts":"1682614526182"} 2023-04-27 16:55:26,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 38bab30e6c18227e87f7eed5826a9e64, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:26,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 38bab30e6c18227e87f7eed5826a9e64, disabling compactions & flushes 2023-04-27 16:55:26,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:26,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:26,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. after waiting 0 ms 2023-04-27 16:55:26,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:26,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 38bab30e6c18227e87f7eed5826a9e64 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:55:26,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/6c0948b2c023485bb437c49d6c504ff0 2023-04-27 16:55:26,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/a58f1823a8374c37a1316ce27c10bf9f 2023-04-27 16:55:26,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf1/6c0948b2c023485bb437c49d6c504ff0 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0 2023-04-27 16:55:26,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:26,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.tmp/cf2/a58f1823a8374c37a1316ce27c10bf9f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f 2023-04-27 16:55:26,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:26,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 38bab30e6c18227e87f7eed5826a9e64 in 37ms, sequenceid=82, compaction requested=true 2023-04-27 16:55:26,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:55:26,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:26,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. 2023-04-27 16:55:26,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 38bab30e6c18227e87f7eed5826a9e64: 2023-04-27 16:55:26,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,379 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=38bab30e6c18227e87f7eed5826a9e64, regionState=CLOSED 2023-04-27 16:55:26,379 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614526379"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614526379"}]},"ts":"1682614526379"} 2023-04-27 16:55:26,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-04-27 16:55:26,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 38bab30e6c18227e87f7eed5826a9e64, server=jenkins-hbase12.apache.org,44953,1682614416269 in 198 msec 2023-04-27 16:55:26,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-04-27 16:55:26,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=38bab30e6c18227e87f7eed5826a9e64, UNASSIGN in 201 msec 2023-04-27 16:55:26,392 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=38bab30e6c18227e87f7eed5826a9e64, threads=14 2023-04-27 16:55:26,392 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,392 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,392 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,392 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,393 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,394 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,402 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8, top=false 2023-04-27 16:55:26,403 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c, top=false 2023-04-27 16:55:26,403 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c, top=false 2023-04-27 16:55:26,403 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e, top=false 2023-04-27 16:55:26,404 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380, top=false 2023-04-27 16:55:26,405 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b, top=false 2023-04-27 16:55:26,406 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0, top=false 2023-04-27 16:55:26,406 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f, top=false 2023-04-27 16:55:26,406 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba, top=false 2023-04-27 16:55:26,407 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b, top=false 2023-04-27 16:55:26,408 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3, top=false 2023-04-27 16:55:26,408 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f, top=false 2023-04-27 16:55:26,409 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa, top=false 2023-04-27 16:55:26,412 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,413 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4, top=false 2023-04-27 16:55:26,418 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,426 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,429 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,434 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8 for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,436 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,436 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,438 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,440 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,440 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,444 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,447 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380 for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3 for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4 for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,451 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0 for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,455 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/.splits/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba for child: 6871275d65296b087ab904889627a9f0, parent: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,456 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,456 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,457 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,458 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,459 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,459 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,460 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380 for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,460 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba for region: 38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:26,460 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 38bab30e6c18227e87f7eed5826a9e64 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-04-27 16:55:26,494 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:26,496 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:26,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1682614526497"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1682614526497"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1682614526497"}]},"ts":"1682614526497"} 2023-04-27 16:55:26,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526497"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614526497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614526497"}]},"ts":"1682614526497"} 2023-04-27 16:55:26,498 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526497"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614526497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614526497"}]},"ts":"1682614526497"} 2023-04-27 16:55:26,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, ASSIGN}] 2023-04-27 16:55:26,502 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, ASSIGN 2023-04-27 16:55:26,502 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=false 2023-04-27 16:55:26,503 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, ASSIGN 2023-04-27 16:55:26,503 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,44953,1682614416269; forceNewPlan=false, retain=false 2023-04-27 16:55:26,652 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:55:26,654 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=18300bad48aae2ec28b560219c1ee82d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526654"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614526654"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614526654"}]},"ts":"1682614526654"} 2023-04-27 16:55:26,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 18300bad48aae2ec28b560219c1ee82d, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:26,658 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6871275d65296b087ab904889627a9f0, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,658 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526658"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614526658"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614526658"}]},"ts":"1682614526658"} 2023-04-27 16:55:26,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 6871275d65296b087ab904889627a9f0, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:26,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:26,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 18300bad48aae2ec28b560219c1ee82d, NAME => 'testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.', STARTKEY => '71', ENDKEY => ''} 2023-04-27 16:55:26,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:26,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:26,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,811 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,812 DEBUG [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf1 2023-04-27 16:55:26,813 DEBUG [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf1 2023-04-27 16:55:26,813 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18300bad48aae2ec28b560219c1ee82d columnFamilyName cf1 2023-04-27 16:55:26,813 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] regionserver.HStore(310): Store=18300bad48aae2ec28b560219c1ee82d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:26,813 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,814 DEBUG [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf2 2023-04-27 16:55:26,815 DEBUG [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf2 2023-04-27 16:55:26,815 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18300bad48aae2ec28b560219c1ee82d columnFamilyName cf2 2023-04-27 16:55:26,815 INFO [StoreOpener-18300bad48aae2ec28b560219c1ee82d-1] regionserver.HStore(310): Store=18300bad48aae2ec28b560219c1ee82d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:26,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:26,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:26,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 18300bad48aae2ec28b560219c1ee82d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63097969, jitterRate=-0.059766992926597595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:26,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 18300bad48aae2ec28b560219c1ee82d: 2023-04-27 16:55:26,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d., pid=133, masterSystemTime=1682614526807 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:26,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:26,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6871275d65296b087ab904889627a9f0, NAME => 'testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.', STARTKEY => '', ENDKEY => '71'} 2023-04-27 16:55:26,822 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=18300bad48aae2ec28b560219c1ee82d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,822 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526822"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614526822"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614526822"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614526822"}]},"ts":"1682614526822"} 2023-04-27 16:55:26,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,824 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-04-27 16:55:26,825 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1 2023-04-27 16:55:26,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 18300bad48aae2ec28b560219c1ee82d, server=jenkins-hbase12.apache.org,44953,1682614416269 in 167 msec 2023-04-27 16:55:26,825 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1 2023-04-27 16:55:26,825 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6871275d65296b087ab904889627a9f0 columnFamilyName cf1 2023-04-27 16:55:26,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, ASSIGN in 324 msec 2023-04-27 16:55:26,830 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba 2023-04-27 16:55:26,833 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b 2023-04-27 16:55:26,836 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0 2023-04-27 16:55:26,839 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4 2023-04-27 16:55:26,841 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3 2023-04-27 16:55:26,843 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b 2023-04-27 16:55:26,845 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f 2023-04-27 16:55:26,846 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(310): Store=6871275d65296b087ab904889627a9f0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:26,846 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,846 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2 2023-04-27 16:55:26,846 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2 2023-04-27 16:55:26,847 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6871275d65296b087ab904889627a9f0 columnFamilyName cf2 2023-04-27 16:55:26,851 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380 2023-04-27 16:55:26,854 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e 2023-04-27 16:55:26,856 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f 2023-04-27 16:55:26,858 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c 2023-04-27 16:55:26,861 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8 2023-04-27 16:55:26,863 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa 2023-04-27 16:55:26,866 DEBUG [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c 2023-04-27 16:55:26,866 INFO [StoreOpener-6871275d65296b087ab904889627a9f0-1] regionserver.HStore(310): Store=6871275d65296b087ab904889627a9f0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:26,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:26,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:26,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 6871275d65296b087ab904889627a9f0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72149387, jitterRate=0.07510964572429657}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:26,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 6871275d65296b087ab904889627a9f0: 2023-04-27 16:55:26,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0., pid=134, masterSystemTime=1682614526807 2023-04-27 16:55:26,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:26,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HStore(1912): 6871275d65296b087ab904889627a9f0/cf2 is initiating minor compaction (all files) 2023-04-27 16:55:26,872 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HStore(1912): 6871275d65296b087ab904889627a9f0/cf1 is initiating minor compaction (all files) 2023-04-27 16:55:26,872 INFO [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6871275d65296b087ab904889627a9f0/cf1 in testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:26,872 INFO [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6871275d65296b087ab904889627a9f0/cf2 in testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:26,872 INFO [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/.tmp, totalSize=34.9 K 2023-04-27 16:55:26,872 INFO [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/.tmp, totalSize=34.9 K 2023-04-27 16:55:26,873 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614524250 2023-04-27 16:55:26,873 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614524250 2023-04-27 16:55:26,873 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614524490 2023-04-27 16:55:26,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:26,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:26,873 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614524490 2023-04-27 16:55:26,873 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6871275d65296b087ab904889627a9f0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614524730 2023-04-27 16:55:26,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614526873"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614526873"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614526873"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614526873"}]},"ts":"1682614526873"} 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614524730 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614524961 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614524961 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614525231 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614525231 2023-04-27 16:55:26,874 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614525918 2023-04-27 16:55:26,875 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614525918 2023-04-27 16:55:26,875 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614526171 2023-04-27 16:55:26,875 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614526171 2023-04-27 16:55:26,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-04-27 16:55:26,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 6871275d65296b087ab904889627a9f0, server=jenkins-hbase12.apache.org,44953,1682614416269 in 216 msec 2023-04-27 16:55:26,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-04-27 16:55:26,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, ASSIGN in 376 msec 2023-04-27 16:55:26,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=38bab30e6c18227e87f7eed5826a9e64, daughterA=6871275d65296b087ab904889627a9f0, daughterB=18300bad48aae2ec28b560219c1ee82d in 701 msec 2023-04-27 16:55:26,890 INFO [RS:0;jenkins-hbase12:44953-longCompactions-0] throttle.PressureAwareThroughputController(145): 6871275d65296b087ab904889627a9f0#cf2#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:26,890 INFO [RS:0;jenkins-hbase12:44953-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6871275d65296b087ab904889627a9f0#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:26,909 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/.tmp/cf2/8a54017cc3c04307918bf5fad4a16329 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/8a54017cc3c04307918bf5fad4a16329 2023-04-27 16:55:26,913 INFO [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6871275d65296b087ab904889627a9f0/cf2 of 6871275d65296b087ab904889627a9f0 into 8a54017cc3c04307918bf5fad4a16329(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:26,913 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6871275d65296b087ab904889627a9f0: 2023-04-27 16:55:26,913 INFO [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0., storeName=6871275d65296b087ab904889627a9f0/cf2, priority=9, startTime=1682614526871; duration=0sec 2023-04-27 16:55:26,914 DEBUG [RS:0;jenkins-hbase12:44953-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:26,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44953] ipc.CallRunner(144): callId: 1033 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:40346 deadline: 1682614586930, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64. is not online on jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:27,051 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-04-27 16:55:27,054 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:27,056 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,056 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-27 16:55:27,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614527059"}]},"ts":"1682614527059"} 2023-04-27 16:55:27,060 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-04-27 16:55:27,102 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid c3e9854d-f6b7-4cf1-bd29-59caf4745df9) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:27,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-27 16:55:27,173 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-04-27 16:55:27,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, UNASSIGN}] 2023-04-27 16:55:27,175 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, UNASSIGN 2023-04-27 16:55:27,175 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=18300bad48aae2ec28b560219c1ee82d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:27,175 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614527175"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614527175"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614527175"}]},"ts":"1682614527175"} 2023-04-27 16:55:27,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 18300bad48aae2ec28b560219c1ee82d, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:27,179 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, UNASSIGN 2023-04-27 16:55:27,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6871275d65296b087ab904889627a9f0, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:27,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614527179"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614527179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614527179"}]},"ts":"1682614527179"} 2023-04-27 16:55:27,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 6871275d65296b087ab904889627a9f0, server=jenkins-hbase12.apache.org,44953,1682614416269}] 2023-04-27 16:55:27,311 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/.tmp/cf1/a0364f898a8a44868b122a1ea94d2562 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/a0364f898a8a44868b122a1ea94d2562 2023-04-27 16:55:27,315 INFO [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6871275d65296b087ab904889627a9f0/cf1 of 6871275d65296b087ab904889627a9f0 into a0364f898a8a44868b122a1ea94d2562(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:27,315 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6871275d65296b087ab904889627a9f0: 2023-04-27 16:55:27,315 INFO [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0., storeName=6871275d65296b087ab904889627a9f0/cf1, priority=9, startTime=1682614526871; duration=0sec 2023-04-27 16:55:27,315 DEBUG [RS:0;jenkins-hbase12:44953-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:27,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:27,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 18300bad48aae2ec28b560219c1ee82d, disabling compactions & flushes 2023-04-27 16:55:27,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:27,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:27,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. after waiting 0 ms 2023-04-27 16:55:27,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:27,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-04-27 16:55:27,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:27,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d. 2023-04-27 16:55:27,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 18300bad48aae2ec28b560219c1ee82d: 2023-04-27 16:55:27,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:27,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:27,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 6871275d65296b087ab904889627a9f0, disabling compactions & flushes 2023-04-27 16:55:27,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:27,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:27,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. after waiting 0 ms 2023-04-27 16:55:27,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:27,335 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=18300bad48aae2ec28b560219c1ee82d, regionState=CLOSED 2023-04-27 16:55:27,336 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614527335"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614527335"}]},"ts":"1682614527335"} 2023-04-27 16:55:27,336 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0] to archive 2023-04-27 16:55:27,337 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:27,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-04-27 16:55:27,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 18300bad48aae2ec28b560219c1ee82d, server=jenkins-hbase12.apache.org,44953,1682614416269 in 160 msec 2023-04-27 16:55:27,338 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-56b533a1cbca4b40a6d2e3f787d1860b 2023-04-27 16:55:27,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=18300bad48aae2ec28b560219c1ee82d, UNASSIGN in 164 msec 2023-04-27 16:55:27,339 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-e11b153c1bf84a24b2fe8cf3f970527b 2023-04-27 16:55:27,340 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-fb06af5fbe9041f9870c57aebd51490f 2023-04-27 16:55:27,341 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-de2833a7f72a4328b2ae095205ed07d3 2023-04-27 16:55:27,342 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-709a8a4ea18c48408ab7a652fd1a15d4 2023-04-27 16:55:27,343 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-06e02cc570c34437b8db95a2b57aa7ba 2023-04-27 16:55:27,344 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-6c0948b2c023485bb437c49d6c504ff0 2023-04-27 16:55:27,345 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f] to archive 2023-04-27 16:55:27,346 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:27,347 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-8bfe3bc49444479eac4fa9d56e3e5380 2023-04-27 16:55:27,348 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a41dc73a313046eebe48c6030865230e 2023-04-27 16:55:27,349 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-cf25508341ba4eccb25f6957be766eaa 2023-04-27 16:55:27,349 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-c6300601a8c94c058b37924fa37aefb8 2023-04-27 16:55:27,350 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-baf010430fb349cda15664887e1bba8c 2023-04-27 16:55:27,351 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-d114e522b1ee41c7a68dc608437c266c 2023-04-27 16:55:27,352 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/testSplitTableRegionEmptyDaughter=38bab30e6c18227e87f7eed5826a9e64-a58f1823a8374c37a1316ce27c10bf9f 2023-04-27 16:55:27,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:55:27,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:27,355 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0. 2023-04-27 16:55:27,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 6871275d65296b087ab904889627a9f0: 2023-04-27 16:55:27,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 6871275d65296b087ab904889627a9f0 2023-04-27 16:55:27,357 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6871275d65296b087ab904889627a9f0, regionState=CLOSED 2023-04-27 16:55:27,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1682614527357"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614527357"}]},"ts":"1682614527357"} 2023-04-27 16:55:27,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-04-27 16:55:27,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 6871275d65296b087ab904889627a9f0, server=jenkins-hbase12.apache.org,44953,1682614416269 in 178 msec 2023-04-27 16:55:27,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-27 16:55:27,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-04-27 16:55:27,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6871275d65296b087ab904889627a9f0, UNASSIGN in 186 msec 2023-04-27 16:55:27,362 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614527362"}]},"ts":"1682614527362"} 2023-04-27 16:55:27,363 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-04-27 16:55:27,373 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-04-27 16:55:27,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 318 msec 2023-04-27 16:55:27,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-27 16:55:27,662 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-04-27 16:55:27,663 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,665 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,666 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-04-27 16:55:27,669 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:27,669 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:27,669 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0 2023-04-27 16:55:27,671 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/recovered.edits] 2023-04-27 16:55:27,671 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/recovered.edits] 2023-04-27 16:55:27,671 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/recovered.edits] 2023-04-27 16:55:27,674 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/a0364f898a8a44868b122a1ea94d2562 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf1/a0364f898a8a44868b122a1ea94d2562 2023-04-27 16:55:27,674 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/06e02cc570c34437b8db95a2b57aa7ba 2023-04-27 16:55:27,675 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/56b533a1cbca4b40a6d2e3f787d1860b 2023-04-27 16:55:27,676 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/8a54017cc3c04307918bf5fad4a16329 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/cf2/8a54017cc3c04307918bf5fad4a16329 2023-04-27 16:55:27,676 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/recovered.edits/88.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d/recovered.edits/88.seqid 2023-04-27 16:55:27,676 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/6c0948b2c023485bb437c49d6c504ff0 2023-04-27 16:55:27,676 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/18300bad48aae2ec28b560219c1ee82d 2023-04-27 16:55:27,677 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/709a8a4ea18c48408ab7a652fd1a15d4 2023-04-27 16:55:27,678 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0/recovered.edits/90.seqid 2023-04-27 16:55:27,678 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/de2833a7f72a4328b2ae095205ed07d3 2023-04-27 16:55:27,678 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/6871275d65296b087ab904889627a9f0 2023-04-27 16:55:27,679 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/e11b153c1bf84a24b2fe8cf3f970527b 2023-04-27 16:55:27,680 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf1/fb06af5fbe9041f9870c57aebd51490f 2023-04-27 16:55:27,682 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/8bfe3bc49444479eac4fa9d56e3e5380 2023-04-27 16:55:27,683 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a41dc73a313046eebe48c6030865230e 2023-04-27 16:55:27,684 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/a58f1823a8374c37a1316ce27c10bf9f 2023-04-27 16:55:27,684 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/baf010430fb349cda15664887e1bba8c 2023-04-27 16:55:27,685 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/c6300601a8c94c058b37924fa37aefb8 2023-04-27 16:55:27,686 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/cf25508341ba4eccb25f6957be766eaa 2023-04-27 16:55:27,687 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/cf2/d114e522b1ee41c7a68dc608437c266c 2023-04-27 16:55:27,689 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64/recovered.edits/85.seqid 2023-04-27 16:55:27,690 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegionEmptyDaughter/38bab30e6c18227e87f7eed5826a9e64 2023-04-27 16:55:27,690 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-04-27 16:55:27,691 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,693 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-04-27 16:55:27,695 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-04-27 16:55:27,695 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,695 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-04-27 16:55:27,696 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614527695"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:27,696 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614527695"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:27,696 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614527695"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:27,697 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:55:27,697 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 38bab30e6c18227e87f7eed5826a9e64, NAME => 'testSplitTableRegionEmptyDaughter,,1682614523246.38bab30e6c18227e87f7eed5826a9e64.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6871275d65296b087ab904889627a9f0, NAME => 'testSplitTableRegionEmptyDaughter,,1682614526178.6871275d65296b087ab904889627a9f0.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 18300bad48aae2ec28b560219c1ee82d, NAME => 'testSplitTableRegionEmptyDaughter,71,1682614526178.18300bad48aae2ec28b560219c1ee82d.', STARTKEY => '71', ENDKEY => ''}] 2023-04-27 16:55:27,697 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-04-27 16:55:27,697 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614527697"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:27,698 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-04-27 16:55:27,710 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-27 16:55:27,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 47 msec 2023-04-27 16:55:27,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-04-27 16:55:27,767 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-04-27 16:55:27,779 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 374), OpenFileDescriptor=842 (was 842), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 197) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=11645 (was 11649) 2023-04-27 16:55:27,791 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=171, AvailableMemoryMB=11645 2023-04-27 16:55:27,792 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:27,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-27 16:55:27,805 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-04-27 16:55:27,806 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-04-27 16:55:27,808 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-04-27 16:55:27,808 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-27 16:55:27,810 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:27,810 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 empty. 2023-04-27 16:55:27,811 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:27,811 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-04-27 16:55:27,967 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-04-27 16:55:27,968 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => da975f9cf22237ce406e56efed890a88, NAME => 'testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing da975f9cf22237ce406e56efed890a88, disabling compactions & flushes 2023-04-27 16:55:27,977 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. after waiting 0 ms 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:27,977 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:27,977 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:27,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-04-27 16:55:27,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614527981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614527981"}]},"ts":"1682614527981"} 2023-04-27 16:55:27,982 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-27 16:55:27,983 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-27 16:55:27,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614527983"}]},"ts":"1682614527983"} 2023-04-27 16:55:27,984 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-04-27 16:55:28,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-04-27 16:55:28,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-27 16:55:28,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-27 16:55:28,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-27 16:55:28,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-27 16:55:28,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, ASSIGN}] 2023-04-27 16:55:28,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, ASSIGN 2023-04-27 16:55:28,008 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:28,159 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-27 16:55:28,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=da975f9cf22237ce406e56efed890a88, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614528160"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614528160"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614528160"}]},"ts":"1682614528160"} 2023-04-27 16:55:28,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure da975f9cf22237ce406e56efed890a88, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:28,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => da975f9cf22237ce406e56efed890a88, NAME => 'testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.', STARTKEY => '', ENDKEY => ''} 2023-04-27 16:55:28,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:28,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:28,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,318 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,320 DEBUG [StoreOpener-da975f9cf22237ce406e56efed890a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1 2023-04-27 16:55:28,320 DEBUG [StoreOpener-da975f9cf22237ce406e56efed890a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1 2023-04-27 16:55:28,320 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da975f9cf22237ce406e56efed890a88 columnFamilyName cf1 2023-04-27 16:55:28,320 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] regionserver.HStore(310): Store=da975f9cf22237ce406e56efed890a88/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:28,320 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,322 DEBUG [StoreOpener-da975f9cf22237ce406e56efed890a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2 2023-04-27 16:55:28,322 DEBUG [StoreOpener-da975f9cf22237ce406e56efed890a88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2 2023-04-27 16:55:28,322 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da975f9cf22237ce406e56efed890a88 columnFamilyName cf2 2023-04-27 16:55:28,322 INFO [StoreOpener-da975f9cf22237ce406e56efed890a88-1] regionserver.HStore(310): Store=da975f9cf22237ce406e56efed890a88/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:28,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:28,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:28,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-27 16:55:28,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened da975f9cf22237ce406e56efed890a88; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72722416, jitterRate=0.0836484432220459}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:28,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:28,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88., pid=143, masterSystemTime=1682614528313 2023-04-27 16:55:28,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=da975f9cf22237ce406e56efed890a88, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614528330"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614528330"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614528330"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614528330"}]},"ts":"1682614528330"} 2023-04-27 16:55:28,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-04-27 16:55:28,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure da975f9cf22237ce406e56efed890a88, server=jenkins-hbase12.apache.org,41143,1682614416328 in 169 msec 2023-04-27 16:55:28,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-04-27 16:55:28,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, ASSIGN in 327 msec 2023-04-27 16:55:28,334 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-27 16:55:28,334 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614528334"}]},"ts":"1682614528334"} 2023-04-27 16:55:28,335 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-04-27 16:55:28,353 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-04-27 16:55:28,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 548 msec 2023-04-27 16:55:28,563 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:28,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:28,565 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:28,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:28,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:28,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,565 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,615 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,615 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,615 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,615 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,616 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,616 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:28,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:28,617 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:28,617 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,617 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:28,618 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,618 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,618 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,619 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-27 16:55:28,625 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,627 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6b3d5e9d9d1147379c24993016f6ca35 2023-04-27 16:55:28,636 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,636 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,636 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,636 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,636 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,640 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/68ec72aa5dd94f22941a25b3579f0f64 2023-04-27 16:55:28,644 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6b3d5e9d9d1147379c24993016f6ca35 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35 2023-04-27 16:55:28,647 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:28,648 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/68ec72aa5dd94f22941a25b3579f0f64 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64 2023-04-27 16:55:28,651 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:28,651 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 32ms, sequenceid=6, compaction requested=false 2023-04-27 16:55:28,652 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-04-27 16:55:28,652 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:28,652 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,652 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:28,652 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:28,652 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,652 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,652 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,657 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,657 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,657 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,658 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:28,659 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c27cb84[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:28,659 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,699 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,699 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,699 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:28,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,699 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:28,700 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:28,720 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,720 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,720 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,720 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,731 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,731 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,731 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,741 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,742 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:28,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:28,745 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:28,745 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:28,745 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:28,745 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:28,745 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,753 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,753 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,754 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,755 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,755 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,763 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:28,763 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:28,763 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,762 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,763 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,764 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,863 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:28,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:28,873 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:28,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:28,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:28,875 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:28,875 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,875 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:28,875 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:28,875 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,875 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,887 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,887 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,887 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:28,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:28,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:28,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:28,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:28,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:28,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:28,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,889 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,890 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:28,890 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:28,890 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,890 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:28,890 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:28,899 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,899 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,899 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,899 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,900 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,899 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,901 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e 2023-04-27 16:55:28,911 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/2d77c1ebff3f4b6b914223b221dde9f1 2023-04-27 16:55:28,914 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e 2023-04-27 16:55:28,918 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:28,919 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/2d77c1ebff3f4b6b914223b221dde9f1 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1 2023-04-27 16:55:28,922 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1, entries=10, sequenceid=19, filesize=5.0 K 2023-04-27 16:55:28,923 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 33ms, sequenceid=19, compaction requested=false 2023-04-27 16:55:28,923 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:28,923 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:28,923 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:28,923 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:28,923 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,923 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:28,923 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:28,931 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,931 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,931 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,931 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:28,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@553462e0[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:28,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:28,933 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:28,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,942 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:28,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:28,942 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:28,952 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,952 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,952 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,952 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,962 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,962 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:28,962 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,962 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:28,963 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,963 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:28,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:28,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:28,965 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:28,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:28,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:28,965 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:28,965 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:28,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,973 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:28,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,973 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:28,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,974 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:28,974 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:28,975 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:28,976 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:28,976 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:28,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:28,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,016 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:29,016 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,016 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,016 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,017 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,117 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:29,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:29,127 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:29,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:29,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:29,129 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:29,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:29,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:29,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,129 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,150 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,150 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,150 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,150 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:29,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,151 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:29,152 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:29,152 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:29,152 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:29,152 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:29,161 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6caad07975aa41ba9872b0569f418f5c 2023-04-27 16:55:29,162 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,173 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,173 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:29,173 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,173 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,173 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:29,576 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/3a27db265f9d46c198cd982a5c078cca 2023-04-27 16:55:29,580 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6caad07975aa41ba9872b0569f418f5c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c 2023-04-27 16:55:29,583 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:29,584 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/3a27db265f9d46c198cd982a5c078cca as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca 2023-04-27 16:55:29,587 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca, entries=10, sequenceid=32, filesize=5.0 K 2023-04-27 16:55:29,587 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 435ms, sequenceid=32, compaction requested=true 2023-04-27 16:55:29,587 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:29,587 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:29,588 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:29,588 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:29,588 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,588 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:29,588 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:29,636 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,636 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:29,637 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,637 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:29,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:29,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:29,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20906d3b[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:29,639 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:29,639 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:29,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:29,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,647 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,647 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:29,647 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:29,648 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:29,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:29,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:29,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:29,668 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:29,668 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:29,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:29,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:29,671 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:29,671 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:29,671 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:29,671 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:29,671 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:29,678 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,678 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:29,678 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,678 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:29,678 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,679 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,679 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,680 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,680 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:29,681 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,681 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,682 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,689 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:29,689 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,689 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:29,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:29,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,689 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:29,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:29,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:29,690 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,789 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:29,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:29,800 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:29,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:29,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:29,801 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:29,801 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,801 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:29,802 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:29,802 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,802 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,941 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,941 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:29,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:29,942 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:29,942 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:29,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:29,944 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:29,944 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:29,943 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:29,944 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:29,944 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:29,944 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:29,944 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:29,994 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,069 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,070 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:30,069 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,070 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,070 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:30,101 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 6c9ebd47-0a00-4e93-9558-ec52eb9cbf04) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:30,354 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/1fa34ce79ec9461db8de648435ecdc39 2023-04-27 16:55:30,370 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/ecad9f72f216492f878ea4f8d8592113 2023-04-27 16:55:30,374 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/1fa34ce79ec9461db8de648435ecdc39 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39 2023-04-27 16:55:30,377 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:30,378 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/ecad9f72f216492f878ea4f8d8592113 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113 2023-04-27 16:55:30,381 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113, entries=10, sequenceid=45, filesize=5.0 K 2023-04-27 16:55:30,382 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 438ms, sequenceid=45, compaction requested=true 2023-04-27 16:55:30,382 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:30,382 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:30,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:30,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:30,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:30,382 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:30,425 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,425 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:30,426 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,426 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:30,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:30,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19b9d194[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:30,428 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:30,428 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,436 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,437 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:30,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:30,436 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:30,436 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:30,437 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:30,446 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:30,446 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:30,447 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:30,447 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:30,447 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:30,447 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:30,490 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:30,490 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,490 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:30,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:30,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:30,490 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:30,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:30,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:30,493 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:30,493 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:30,493 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:30,493 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:30,493 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,510 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:30,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,511 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,511 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,512 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:30,513 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:30,513 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,520 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,520 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:30,521 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:30,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,520 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:30,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,521 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,621 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:30,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:30,632 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:30,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:30,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:30,633 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:30,633 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:30,633 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:30,633 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:30,634 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,634 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,647 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:30,647 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:30,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:30,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:30,648 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:30,648 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:30,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:30,648 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:30,649 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:30,649 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:30,649 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:30,649 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:30,656 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/e8e42583c37b47f0834e81bb6e5c3b7b 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:30,657 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:31,066 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/5e041d2b272a4415b4bca93d20c3dd9f 2023-04-27 16:55:31,071 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/e8e42583c37b47f0834e81bb6e5c3b7b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b 2023-04-27 16:55:31,075 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:31,075 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/5e041d2b272a4415b4bca93d20c3dd9f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f 2023-04-27 16:55:31,082 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f, entries=10, sequenceid=58, filesize=5.0 K 2023-04-27 16:55:31,082 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 433ms, sequenceid=58, compaction requested=true 2023-04-27 16:55:31,082 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:31,082 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:31,082 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:31,082 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:31,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:31,083 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:31,099 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,099 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,099 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,100 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:31,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b2052cc[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:31,103 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:31,103 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,110 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,110 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,110 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:31,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,110 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,111 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,111 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:31,111 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,111 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,111 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:31,120 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,120 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,120 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,131 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,131 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,131 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,141 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,141 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,141 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,142 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:31,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:31,144 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:31,144 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:31,144 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:31,144 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:31,144 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,153 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,154 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-04-27 16:55:31,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,162 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:31,163 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,162 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,162 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,163 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,164 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,263 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:31,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:31,276 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-04-27 16:55:31,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-27 16:55:31,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-27 16:55:31,278 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-27 16:55:31,278 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:31,278 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-27 16:55:31,278 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-27 16:55:31,278 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,278 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,308 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,308 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,308 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-27 16:55:31,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-27 16:55:31,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:31,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:31,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:31,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:31,309 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46011,1682614416374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-27 16:55:31,310 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-27 16:55:31,310 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. started... 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:31,310 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,44953,1682614416269' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:31,310 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-27 16:55:31,320 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6067d165cdad4eed89f2ce2eba010d9d 2023-04-27 16:55:31,346 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,362 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,362 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,362 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:31,363 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,363 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:31,733 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/1922071990ff438399c796d3477d481f 2023-04-27 16:55:31,737 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/6067d165cdad4eed89f2ce2eba010d9d as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d 2023-04-27 16:55:31,740 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:31,740 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/1922071990ff438399c796d3477d481f as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f 2023-04-27 16:55:31,745 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f, entries=10, sequenceid=71, filesize=5.0 K 2023-04-27 16:55:31,746 INFO [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 436ms, sequenceid=71, compaction requested=true 2023-04-27 16:55:31,746 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:31,746 DEBUG [rs(jenkins-hbase12.apache.org,41143,1682614416328)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:31,746 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-27 16:55:31,746 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-27 16:55:31,746 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,746 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-27 16:55:31,746 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-27 16:55:31,794 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,794 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,794 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,794 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-27 16:55:31,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41143,1682614416328' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-27 16:55:31,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d751c2a[Count = 0] remaining members to acquire global barrier 2023-04-27 16:55:31,796 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-27 16:55:31,796 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,805 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,805 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-27 16:55:31,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,44953,1682614416269' in zk 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46011,1682614416374' in zk 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-27 16:55:31,805 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41143,1682614416328' in zk 2023-04-27 16:55:31,815 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,815 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,815 DEBUG [member: 'jenkins-hbase12.apache.org,44953,1682614416269' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,825 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,825 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,826 DEBUG [member: 'jenkins-hbase12.apache.org,41143,1682614416328' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,825 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,825 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-27 16:55:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,826 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,826 DEBUG [member: 'jenkins-hbase12.apache.org,46011,1682614416374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-27 16:55:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,41143,1682614416328': 2023-04-27 16:55:31,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41143,1682614416328' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-27 16:55:31,829 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-27 16:55:31,829 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-27 16:55:31,829 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-27 16:55:31,829 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-27 16:55:31,829 INFO [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-27 16:55:31,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,836 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-27 16:55:31,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,836 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,837 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,837 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,838 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-27 16:55:31,838 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-27 16:55:31,838 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-27 16:55:31,847 DEBUG [(jenkins-hbase12.apache.org,37667,1682614415067)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,847 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-27 16:55:31,847 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-27 16:55:31,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,846 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-27 16:55:31,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-27 16:55:31,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-27 16:55:31,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-27 16:55:31,847 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-27 16:55:31,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-27 16:55:31,848 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-27 16:55:31,879 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-27 16:55:31,947 DEBUG [Listener at localhost.localdomain/45365] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-27 16:55:31,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-27 16:55:31,955 DEBUG [Listener at localhost.localdomain/45365] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,955 DEBUG [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=da975f9cf22237ce406e56efed890a88, daughterA=517717b21e44640ce4cfb7104e350f73, daughterB=86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:31,956 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=da975f9cf22237ce406e56efed890a88, daughterA=517717b21e44640ce4cfb7104e350f73, daughterB=86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:31,956 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=da975f9cf22237ce406e56efed890a88, daughterA=517717b21e44640ce4cfb7104e350f73, daughterB=86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:31,956 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=da975f9cf22237ce406e56efed890a88, daughterA=517717b21e44640ce4cfb7104e350f73, daughterB=86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:31,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, UNASSIGN}] 2023-04-27 16:55:31,958 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, UNASSIGN 2023-04-27 16:55:31,959 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=da975f9cf22237ce406e56efed890a88, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:31,959 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614531959"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614531959"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614531959"}]},"ts":"1682614531959"} 2023-04-27 16:55:31,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure da975f9cf22237ce406e56efed890a88, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:32,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing da975f9cf22237ce406e56efed890a88, disabling compactions & flushes 2023-04-27 16:55:32,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:32,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:32,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. after waiting 0 ms 2023-04-27 16:55:32,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:32,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing da975f9cf22237ce406e56efed890a88 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-27 16:55:32,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 2023-04-27 16:55:32,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/2f220c53298349088510325313414976 2023-04-27 16:55:32,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 2023-04-27 16:55:32,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:32,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/.tmp/cf2/2f220c53298349088510325313414976 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976 2023-04-27 16:55:32,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976, entries=8, sequenceid=82, filesize=5.0 K 2023-04-27 16:55:32,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for da975f9cf22237ce406e56efed890a88 in 39ms, sequenceid=82, compaction requested=true 2023-04-27 16:55:32,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-27 16:55:32,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:32,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. 2023-04-27 16:55:32,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for da975f9cf22237ce406e56efed890a88: 2023-04-27 16:55:32,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=da975f9cf22237ce406e56efed890a88, regionState=CLOSED 2023-04-27 16:55:32,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614532161"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614532161"}]},"ts":"1682614532161"} 2023-04-27 16:55:32,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-04-27 16:55:32,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure da975f9cf22237ce406e56efed890a88, server=jenkins-hbase12.apache.org,41143,1682614416328 in 203 msec 2023-04-27 16:55:32,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-04-27 16:55:32,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=da975f9cf22237ce406e56efed890a88, UNASSIGN in 206 msec 2023-04-27 16:55:32,179 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=da975f9cf22237ce406e56efed890a88, threads=14 2023-04-27 16:55:32,179 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,179 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,179 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,179 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,180 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,180 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,180 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,180 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,180 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,185 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,185 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,188 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,188 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,189 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,396 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,399 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,431 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,431 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,444 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,456 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,469 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,476 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,485 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,487 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,487 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,763 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,764 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,803 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976 for region: da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:32,804 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region da975f9cf22237ce406e56efed890a88 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-27 16:55:32,838 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:32,839 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-27 16:55:32,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1682614532840"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1682614532840"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1682614532840"}]},"ts":"1682614532840"} 2023-04-27 16:55:32,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614532840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614532840"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614532840"}]},"ts":"1682614532840"} 2023-04-27 16:55:32,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614532840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614532840"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614532840"}]},"ts":"1682614532840"} 2023-04-27 16:55:32,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, ASSIGN}] 2023-04-27 16:55:32,843 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, ASSIGN 2023-04-27 16:55:32,844 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:32,844 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, ASSIGN 2023-04-27 16:55:32,844 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41143,1682614416328; forceNewPlan=false, retain=false 2023-04-27 16:55:32,994 INFO [jenkins-hbase12:37667] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-27 16:55:32,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=517717b21e44640ce4cfb7104e350f73, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:32,995 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614532995"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614532995"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614532995"}]},"ts":"1682614532995"} 2023-04-27 16:55:32,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 517717b21e44640ce4cfb7104e350f73, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:32,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=86e8638e4127a2a723807c5377459a62, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:32,999 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614532999"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614532999"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614532999"}]},"ts":"1682614532999"} 2023-04-27 16:55:33,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 86e8638e4127a2a723807c5377459a62, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:33,103 ERROR [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 5799fd8f-0a4b-4c91-a1ff-1fe687f21837) service to localhost.localdomain/127.0.0.1:43755 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-27 16:55:33,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 517717b21e44640ce4cfb7104e350f73, NAME => 'testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.', STARTKEY => '', ENDKEY => '41'} 2023-04-27 16:55:33,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:33,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:33,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,153 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,154 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1 2023-04-27 16:55:33,154 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1 2023-04-27 16:55:33,154 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517717b21e44640ce4cfb7104e350f73 columnFamilyName cf1 2023-04-27 16:55:33,172 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-bottom 2023-04-27 16:55:33,177 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-bottom 2023-04-27 16:55:33,182 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-bottom 2023-04-27 16:55:33,189 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-bottom 2023-04-27 16:55:33,196 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-bottom 2023-04-27 16:55:33,199 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-bottom 2023-04-27 16:55:33,203 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-bottom 2023-04-27 16:55:33,204 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(310): Store=517717b21e44640ce4cfb7104e350f73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:33,204 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,205 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2 2023-04-27 16:55:33,205 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2 2023-04-27 16:55:33,205 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517717b21e44640ce4cfb7104e350f73 columnFamilyName cf2 2023-04-27 16:55:33,218 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-bottom 2023-04-27 16:55:33,230 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-bottom 2023-04-27 16:55:33,241 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-bottom 2023-04-27 16:55:33,245 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-bottom 2023-04-27 16:55:33,250 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-bottom 2023-04-27 16:55:33,254 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-bottom 2023-04-27 16:55:33,257 DEBUG [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-bottom 2023-04-27 16:55:33,258 INFO [StoreOpener-517717b21e44640ce4cfb7104e350f73-1] regionserver.HStore(310): Store=517717b21e44640ce4cfb7104e350f73/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:33,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:33,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 517717b21e44640ce4cfb7104e350f73; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60694093, jitterRate=-0.09558753669261932}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:33,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 517717b21e44640ce4cfb7104e350f73: 2023-04-27 16:55:33,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73., pid=149, masterSystemTime=1682614533148 2023-04-27 16:55:33,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,264 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:33,264 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:33,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,264 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1912): 517717b21e44640ce4cfb7104e350f73/cf2 is initiating minor compaction (all files) 2023-04-27 16:55:33,264 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): 517717b21e44640ce4cfb7104e350f73/cf1 is initiating minor compaction (all files) 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 517717b21e44640ce4cfb7104e350f73/cf2 in testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 517717b21e44640ce4cfb7104e350f73/cf1 in testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/.tmp, totalSize=34.9 K 2023-04-27 16:55:33,264 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-bottom] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/.tmp, totalSize=34.9 K 2023-04-27 16:55:33,265 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614528559 2023-04-27 16:55:33,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 86e8638e4127a2a723807c5377459a62, NAME => 'testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.', STARTKEY => '41', ENDKEY => ''} 2023-04-27 16:55:33,266 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1682614528559 2023-04-27 16:55:33,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-27 16:55:33,266 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=517717b21e44640ce4cfb7104e350f73, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:33,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,266 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533266"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614533266"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614533266"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614533266"}]},"ts":"1682614533266"} 2023-04-27 16:55:33,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-27 16:55:33,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7894): checking encryption for 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7897): checking classloading for 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,266 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614528865 2023-04-27 16:55:33,267 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1682614528865 2023-04-27 16:55:33,268 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614529118 2023-04-27 16:55:33,268 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1682614529118 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614529791 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1682614529791 2023-04-27 16:55:33,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-04-27 16:55:33,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 517717b21e44640ce4cfb7104e350f73, server=jenkins-hbase12.apache.org,41143,1682614416328 in 270 msec 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614530623 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1682614530623 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614531265 2023-04-27 16:55:33,269 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1682614531265 2023-04-27 16:55:33,270 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614531949 2023-04-27 16:55:33,270 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1682614531949 2023-04-27 16:55:33,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, ASSIGN in 426 msec 2023-04-27 16:55:33,276 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,278 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1 2023-04-27 16:55:33,278 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1 2023-04-27 16:55:33,279 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86e8638e4127a2a723807c5377459a62 columnFamilyName cf1 2023-04-27 16:55:33,279 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 517717b21e44640ce4cfb7104e350f73#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:33,279 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] throttle.PressureAwareThroughputController(145): 517717b21e44640ce4cfb7104e350f73#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:33,299 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-top 2023-04-27 16:55:33,306 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-top 2023-04-27 16:55:33,315 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-top 2023-04-27 16:55:33,319 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/.tmp/cf2/61b391c565164af38b3a907162589dbe as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/61b391c565164af38b3a907162589dbe 2023-04-27 16:55:33,322 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-top 2023-04-27 16:55:33,323 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/.tmp/cf1/52eb5c877f8146098ee7b56d60654140 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/52eb5c877f8146098ee7b56d60654140 2023-04-27 16:55:33,325 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-top 2023-04-27 16:55:33,329 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 517717b21e44640ce4cfb7104e350f73/cf2 of 517717b21e44640ce4cfb7104e350f73 into 61b391c565164af38b3a907162589dbe(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:33,329 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 517717b21e44640ce4cfb7104e350f73: 2023-04-27 16:55:33,329 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73., storeName=517717b21e44640ce4cfb7104e350f73/cf2, priority=9, startTime=1682614533264; duration=0sec 2023-04-27 16:55:33,329 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,333 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-top 2023-04-27 16:55:33,334 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 517717b21e44640ce4cfb7104e350f73/cf1 of 517717b21e44640ce4cfb7104e350f73 into 52eb5c877f8146098ee7b56d60654140(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:33,334 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 517717b21e44640ce4cfb7104e350f73: 2023-04-27 16:55:33,334 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73., storeName=517717b21e44640ce4cfb7104e350f73/cf1, priority=9, startTime=1682614533263; duration=0sec 2023-04-27 16:55:33,334 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,338 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-top 2023-04-27 16:55:33,338 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(310): Store=86e8638e4127a2a723807c5377459a62/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:33,338 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,339 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2 2023-04-27 16:55:33,339 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2 2023-04-27 16:55:33,339 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86e8638e4127a2a723807c5377459a62 columnFamilyName cf2 2023-04-27 16:55:33,355 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-top 2023-04-27 16:55:33,358 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-top 2023-04-27 16:55:33,364 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-top 2023-04-27 16:55:33,367 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-top 2023-04-27 16:55:33,369 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-top 2023-04-27 16:55:33,372 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-top 2023-04-27 16:55:33,379 DEBUG [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-top 2023-04-27 16:55:33,379 INFO [StoreOpener-86e8638e4127a2a723807c5377459a62-1] regionserver.HStore(310): Store=86e8638e4127a2a723807c5377459a62/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-27 16:55:33,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-27 16:55:33,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1055): writing seq id for 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1072): Opened 86e8638e4127a2a723807c5377459a62; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72641087, jitterRate=0.08243654668331146}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-27 16:55:33,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(965): Region open journal for 86e8638e4127a2a723807c5377459a62: 2023-04-27 16:55:33,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62., pid=150, masterSystemTime=1682614533148 2023-04-27 16:55:33,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-27 16:55:33,384 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:33,384 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,384 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1912): 86e8638e4127a2a723807c5377459a62/cf1 is initiating minor compaction (all files) 2023-04-27 16:55:33,384 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-27 16:55:33,384 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 86e8638e4127a2a723807c5377459a62/cf1 in testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,384 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,384 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1912): 86e8638e4127a2a723807c5377459a62/cf2 is initiating minor compaction (all files) 2023-04-27 16:55:33,384 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 86e8638e4127a2a723807c5377459a62/cf2 in testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,385 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/.tmp, totalSize=34.9 K 2023-04-27 16:55:33,385 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-top] into tmpdir=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/.tmp, totalSize=34.9 K 2023-04-27 16:55:33,385 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614528559 2023-04-27 16:55:33,386 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1682614528559 2023-04-27 16:55:33,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,386 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=86e8638e4127a2a723807c5377459a62, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:33,386 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533386"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1682614533386"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1682614533386"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1682614533386"}]},"ts":"1682614533386"} 2023-04-27 16:55:33,388 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614528865 2023-04-27 16:55:33,388 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1682614528865 2023-04-27 16:55:33,388 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614529118 2023-04-27 16:55:33,389 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1682614529118 2023-04-27 16:55:33,389 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614529791 2023-04-27 16:55:33,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-04-27 16:55:33,389 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1682614529791 2023-04-27 16:55:33,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 86e8638e4127a2a723807c5377459a62, server=jenkins-hbase12.apache.org,41143,1682614416328 in 387 msec 2023-04-27 16:55:33,389 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614530623 2023-04-27 16:55:33,390 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1682614530623 2023-04-27 16:55:33,390 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614531265 2023-04-27 16:55:33,390 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] compactions.Compactor(207): Compacting 2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614531949 2023-04-27 16:55:33,390 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1682614531265 2023-04-27 16:55:33,391 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] compactions.Compactor(207): Compacting 9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1682614531949 2023-04-27 16:55:33,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-04-27 16:55:33,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, ASSIGN in 546 msec 2023-04-27 16:55:33,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=da975f9cf22237ce406e56efed890a88, daughterA=517717b21e44640ce4cfb7104e350f73, daughterB=86e8638e4127a2a723807c5377459a62 in 1.4370 sec 2023-04-27 16:55:33,401 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] throttle.PressureAwareThroughputController(145): 86e8638e4127a2a723807c5377459a62#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:33,402 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 86e8638e4127a2a723807c5377459a62#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-27 16:55:33,420 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/.tmp/cf2/06ff6d5350444c179ab4c5e431c9d416 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/06ff6d5350444c179ab4c5e431c9d416 2023-04-27 16:55:33,424 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 86e8638e4127a2a723807c5377459a62/cf2 of 86e8638e4127a2a723807c5377459a62 into 06ff6d5350444c179ab4c5e431c9d416(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:33,424 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 86e8638e4127a2a723807c5377459a62: 2023-04-27 16:55:33,424 INFO [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62., storeName=86e8638e4127a2a723807c5377459a62/cf2, priority=9, startTime=1682614533384; duration=0sec 2023-04-27 16:55:33,424 DEBUG [RS:1;jenkins-hbase12:41143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41143] ipc.CallRunner(144): callId: 1164 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:49980 deadline: 1682614593457, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88. is not online on jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:33,567 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:55:33,572 INFO [Listener at localhost.localdomain/45365] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-27 16:55:33,573 DEBUG [Listener at localhost.localdomain/45365] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-27 16:55:33,588 WARN [Listener at localhost.localdomain/45365] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-27 16:55:33,590 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-04-27 16:55:33,590 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$11(2418): Client=jenkins//136.243.104.168 disable testSplitTableRegion 2023-04-27 16:55:33,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614533592"}]},"ts":"1682614533592"} 2023-04-27 16:55:33,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-27 16:55:33,593 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-04-27 16:55:33,615 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-04-27 16:55:33,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, UNASSIGN}] 2023-04-27 16:55:33,616 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, UNASSIGN 2023-04-27 16:55:33,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=86e8638e4127a2a723807c5377459a62, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:33,617 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533616"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614533616"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614533616"}]},"ts":"1682614533616"} 2023-04-27 16:55:33,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 86e8638e4127a2a723807c5377459a62, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:33,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, UNASSIGN 2023-04-27 16:55:33,619 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=517717b21e44640ce4cfb7104e350f73, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:33,619 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533619"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1682614533619"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1682614533619"}]},"ts":"1682614533619"} 2023-04-27 16:55:33,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 517717b21e44640ce4cfb7104e350f73, server=jenkins-hbase12.apache.org,41143,1682614416328}] 2023-04-27 16:55:33,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-27 16:55:33,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 86e8638e4127a2a723807c5377459a62, disabling compactions & flushes 2023-04-27 16:55:33,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,821 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/.tmp/cf1/d5c68d44be344b929a342f9ba5cf2b47 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/d5c68d44be344b929a342f9ba5cf2b47 2023-04-27 16:55:33,825 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 86e8638e4127a2a723807c5377459a62/cf1 of 86e8638e4127a2a723807c5377459a62 into d5c68d44be344b929a342f9ba5cf2b47(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-27 16:55:33,825 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 86e8638e4127a2a723807c5377459a62: 2023-04-27 16:55:33,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,825 INFO [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62., storeName=86e8638e4127a2a723807c5377459a62/cf1, priority=9, startTime=1682614533383; duration=0sec 2023-04-27 16:55:33,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. after waiting 0 ms 2023-04-27 16:55:33,825 DEBUG [RS:1;jenkins-hbase12:41143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-27 16:55:33,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,826 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-top] to archive 2023-04-27 16:55:33,826 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:33,827 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,828 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,829 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,830 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,830 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,831 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,832 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,834 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-top, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-top] to archive 2023-04-27 16:55:33,834 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:33,835 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,836 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,837 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,838 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,838 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,839 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,840 DEBUG [StoreCloser-testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,842 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:55:33,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:33,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62. 2023-04-27 16:55:33,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 86e8638e4127a2a723807c5377459a62: 2023-04-27 16:55:33,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 517717b21e44640ce4cfb7104e350f73, disabling compactions & flushes 2023-04-27 16:55:33,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=86e8638e4127a2a723807c5377459a62, regionState=CLOSED 2023-04-27 16:55:33,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. after waiting 0 ms 2023-04-27 16:55:33,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614533844"}]},"ts":"1682614533844"} 2023-04-27 16:55:33,845 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8-bottom] to archive 2023-04-27 16:55:33,846 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:33,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-04-27 16:55:33,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 86e8638e4127a2a723807c5377459a62, server=jenkins-hbase12.apache.org,41143,1682614416328 in 229 msec 2023-04-27 16:55:33,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86e8638e4127a2a723807c5377459a62, UNASSIGN in 233 msec 2023-04-27 16:55:33,850 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6b3d5e9d9d1147379c24993016f6ca35.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,851 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,851 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6caad07975aa41ba9872b0569f418f5c.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,852 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/1fa34ce79ec9461db8de648435ecdc39.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,853 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/e8e42583c37b47f0834e81bb6e5c3b7b.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,854 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/6067d165cdad4eed89f2ce2eba010d9d.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,855 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,856 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f-bottom, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88->hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976-bottom] to archive 2023-04-27 16:55:33,857 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-27 16:55:33,858 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/68ec72aa5dd94f22941a25b3579f0f64.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,859 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2d77c1ebff3f4b6b914223b221dde9f1.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,860 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/3a27db265f9d46c198cd982a5c078cca.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,861 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/ecad9f72f216492f878ea4f8d8592113.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,862 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/5e041d2b272a4415b4bca93d20c3dd9f.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,862 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/1922071990ff438399c796d3477d481f.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,863 DEBUG [StoreCloser-testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/2f220c53298349088510325313414976.da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-27 16:55:33,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:33,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73. 2023-04-27 16:55:33,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 517717b21e44640ce4cfb7104e350f73: 2023-04-27 16:55:33,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,868 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=517717b21e44640ce4cfb7104e350f73, regionState=CLOSED 2023-04-27 16:55:33,868 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1682614533868"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1682614533868"}]},"ts":"1682614533868"} 2023-04-27 16:55:33,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-04-27 16:55:33,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 517717b21e44640ce4cfb7104e350f73, server=jenkins-hbase12.apache.org,41143,1682614416328 in 249 msec 2023-04-27 16:55:33,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-04-27 16:55:33,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=517717b21e44640ce4cfb7104e350f73, UNASSIGN in 255 msec 2023-04-27 16:55:33,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1682614533871"}]},"ts":"1682614533871"} 2023-04-27 16:55:33,873 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-04-27 16:55:33,888 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-04-27 16:55:33,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 299 msec 2023-04-27 16:55:33,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-27 16:55:33,894 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-04-27 16:55:33,895 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.HMaster$5(2228): Client=jenkins//136.243.104.168 delete testSplitTableRegion 2023-04-27 16:55:33,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,897 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,898 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-04-27 16:55:33,901 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,901 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,901 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,903 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/recovered.edits] 2023-04-27 16:55:33,903 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/recovered.edits] 2023-04-27 16:55:33,903 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2, FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/recovered.edits] 2023-04-27 16:55:33,905 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/d5c68d44be344b929a342f9ba5cf2b47 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf1/d5c68d44be344b929a342f9ba5cf2b47 2023-04-27 16:55:33,905 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/52eb5c877f8146098ee7b56d60654140 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf1/52eb5c877f8146098ee7b56d60654140 2023-04-27 16:55:33,906 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/1fa34ce79ec9461db8de648435ecdc39 2023-04-27 16:55:33,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6067d165cdad4eed89f2ce2eba010d9d 2023-04-27 16:55:33,908 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/61b391c565164af38b3a907162589dbe to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/cf2/61b391c565164af38b3a907162589dbe 2023-04-27 16:55:33,908 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/06ff6d5350444c179ab4c5e431c9d416 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/cf2/06ff6d5350444c179ab4c5e431c9d416 2023-04-27 16:55:33,908 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6b3d5e9d9d1147379c24993016f6ca35 2023-04-27 16:55:33,909 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/6caad07975aa41ba9872b0569f418f5c 2023-04-27 16:55:33,910 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73/recovered.edits/90.seqid 2023-04-27 16:55:33,910 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/recovered.edits/90.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62/recovered.edits/90.seqid 2023-04-27 16:55:33,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/86e8638e4127a2a723807c5377459a62 2023-04-27 16:55:33,911 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/517717b21e44640ce4cfb7104e350f73 2023-04-27 16:55:33,911 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/7cfa76f6ddcc4c4ba70098f6382f5e0e 2023-04-27 16:55:33,912 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/9cdbe17a6b5142c5a2ac7967af87c6f8 2023-04-27 16:55:33,913 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf1/e8e42583c37b47f0834e81bb6e5c3b7b 2023-04-27 16:55:33,915 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/1922071990ff438399c796d3477d481f 2023-04-27 16:55:33,916 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2d77c1ebff3f4b6b914223b221dde9f1 2023-04-27 16:55:33,917 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/2f220c53298349088510325313414976 2023-04-27 16:55:33,918 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/3a27db265f9d46c198cd982a5c078cca 2023-04-27 16:55:33,919 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/5e041d2b272a4415b4bca93d20c3dd9f 2023-04-27 16:55:33,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/68ec72aa5dd94f22941a25b3579f0f64 2023-04-27 16:55:33,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113 to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/cf2/ecad9f72f216492f878ea4f8d8592113 2023-04-27 16:55:33,922 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/recovered.edits/85.seqid to hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/archive/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88/recovered.edits/85.seqid 2023-04-27 16:55:33,923 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/.tmp/data/default/testSplitTableRegion/da975f9cf22237ce406e56efed890a88 2023-04-27 16:55:33,923 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-04-27 16:55:33,925 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,926 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-04-27 16:55:33,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-04-27 16:55:33,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-04-27 16:55:33,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614533930"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:33,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614533930"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:33,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1682614533930"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:33,933 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-27 16:55:33,933 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => da975f9cf22237ce406e56efed890a88, NAME => 'testSplitTableRegion,,1682614527805.da975f9cf22237ce406e56efed890a88.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 517717b21e44640ce4cfb7104e350f73, NAME => 'testSplitTableRegion,,1682614531955.517717b21e44640ce4cfb7104e350f73.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 86e8638e4127a2a723807c5377459a62, NAME => 'testSplitTableRegion,41,1682614531955.86e8638e4127a2a723807c5377459a62.', STARTKEY => '41', ENDKEY => ''}] 2023-04-27 16:55:33,933 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-04-27 16:55:33,933 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1682614533933"}]},"ts":"9223372036854775807"} 2023-04-27 16:55:33,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-04-27 16:55:33,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-27 16:55:33,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 47 msec 2023-04-27 16:55:33,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37667] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-04-27 16:55:33,999 INFO [Listener at localhost.localdomain/45365] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-04-27 16:55:34,017 INFO [Listener at localhost.localdomain/45365] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 374) - Thread LEAK? -, OpenFileDescriptor=824 (was 842), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 204) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=11978 (was 11645) - AvailableMemoryMB LEAK? - 2023-04-27 16:55:34,017 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-04-27 16:55:34,017 INFO [Listener at localhost.localdomain/45365] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-27 16:55:34,018 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ea999b8 to 127.0.0.1:58499 2023-04-27 16:55:34,018 DEBUG [Listener at localhost.localdomain/45365] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,021 DEBUG [Listener at localhost.localdomain/45365] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-04-27 16:55:34,021 DEBUG [Listener at localhost.localdomain/45365] util.JVMClusterUtil(257): Found active master hash=851527373, stopped=false 2023-04-27 16:55:34,021 INFO [Listener at localhost.localdomain/45365] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:55:34,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-27 16:55:34,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-27 16:55:34,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-27 16:55:34,034 INFO [Listener at localhost.localdomain/45365] procedure2.ProcedureExecutor(629): Stopping 2023-04-27 16:55:34,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-27 16:55:34,034 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:55:34,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:55:34,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:55:34,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:55:34,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-27 16:55:34,035 DEBUG [Listener at localhost.localdomain/45365] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73ce194e to 127.0.0.1:58499 2023-04-27 16:55:34,035 DEBUG [Listener at localhost.localdomain/45365] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,44953,1682614416269' ***** 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,41143,1682614416328' ***** 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,46011,1682614416374' ***** 2023-04-27 16:55:34,035 INFO [Listener at localhost.localdomain/45365] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-27 16:55:34,035 INFO [RS:0;jenkins-hbase12:44953] regionserver.HeapMemoryManager(220): Stopping 2023-04-27 16:55:34,036 INFO [RS:0;jenkins-hbase12:44953] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-27 16:55:34,036 INFO [RS:0;jenkins-hbase12:44953] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-27 16:55:34,036 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:34,036 DEBUG [RS:0;jenkins-hbase12:44953] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0983c9a4 to 127.0.0.1:58499 2023-04-27 16:55:34,036 DEBUG [RS:0;jenkins-hbase12:44953] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,037 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,44953,1682614416269; all regions closed. 2023-04-27 16:55:34,036 INFO [RS:2;jenkins-hbase12:46011] regionserver.HeapMemoryManager(220): Stopping 2023-04-27 16:55:34,036 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-27 16:55:34,036 INFO [RS:1;jenkins-hbase12:41143] regionserver.HeapMemoryManager(220): Stopping 2023-04-27 16:55:34,037 INFO [RS:1;jenkins-hbase12:41143] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-27 16:55:34,037 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-27 16:55:34,037 INFO [RS:2;jenkins-hbase12:46011] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-27 16:55:34,037 INFO [RS:2;jenkins-hbase12:46011] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-27 16:55:34,037 INFO [RS:1;jenkins-hbase12:41143] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-27 16:55:34,037 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(3304): Received CLOSE for a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:55:34,038 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:34,038 DEBUG [RS:1;jenkins-hbase12:41143] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4aa81090 to 127.0.0.1:58499 2023-04-27 16:55:34,038 DEBUG [RS:1;jenkins-hbase12:41143] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,038 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,41143,1682614416328; all regions closed. 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:34,038 DEBUG [RS:2;jenkins-hbase12:46011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71054da7 to 127.0.0.1:58499 2023-04-27 16:55:34,038 DEBUG [RS:2;jenkins-hbase12:46011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-27 16:55:34,038 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-04-27 16:55:34,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing a181d38dc7ff4539b60029e08bf407bf, disabling compactions & flushes 2023-04-27 16:55:34,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:55:34,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:55:34,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. after waiting 0 ms 2023-04-27 16:55:34,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:55:34,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing a181d38dc7ff4539b60029e08bf407bf 1/1 column families, dataSize=78 B heapSize=488 B 2023-04-27 16:55:34,040 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-04-27 16:55:34,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-27 16:55:34,040 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1479): Online Regions={a181d38dc7ff4539b60029e08bf407bf=hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf., 1588230740=hbase:meta,,1.1588230740} 2023-04-27 16:55:34,040 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-27 16:55:34,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-27 16:55:34,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-27 16:55:34,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-27 16:55:34,040 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-04-27 16:55:34,041 DEBUG [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1505): Waiting on 1588230740, a181d38dc7ff4539b60029e08bf407bf 2023-04-27 16:55:34,062 DEBUG [RS:1;jenkins-hbase12:41143] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs 2023-04-27 16:55:34,063 INFO [RS:1;jenkins-hbase12:41143] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C41143%2C1682614416328:(num 1682614418038) 2023-04-27 16:55:34,063 DEBUG [RS:1;jenkins-hbase12:41143] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,063 INFO [RS:1;jenkins-hbase12:41143] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,064 INFO [RS:1;jenkins-hbase12:41143] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-27 16:55:34,064 INFO [RS:1;jenkins-hbase12:41143] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-27 16:55:34,065 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-27 16:55:34,065 INFO [RS:1;jenkins-hbase12:41143] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-27 16:55:34,065 INFO [RS:1;jenkins-hbase12:41143] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-27 16:55:34,067 INFO [RS:1;jenkins-hbase12:41143] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:41143 2023-04-27 16:55:34,070 DEBUG [RS:0;jenkins-hbase12:44953] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs 2023-04-27 16:55:34,070 INFO [RS:0;jenkins-hbase12:44953] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C44953%2C1682614416269:(num 1682614418038) 2023-04-27 16:55:34,070 DEBUG [RS:0;jenkins-hbase12:44953] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,070 INFO [RS:0;jenkins-hbase12:44953] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,070 INFO [RS:0;jenkins-hbase12:44953] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-27 16:55:34,071 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,071 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,071 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,072 INFO [RS:0;jenkins-hbase12:44953] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-27 16:55:34,072 INFO [RS:0;jenkins-hbase12:44953] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-27 16:55:34,072 INFO [RS:0;jenkins-hbase12:44953] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-27 16:55:34,072 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-27 16:55:34,079 INFO [RS:0;jenkins-hbase12:44953] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:44953 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41143,1682614416328 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:55:34,086 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:55:34,088 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@429b4fff rejected from java.util.concurrent.ThreadPoolExecutor@46a4d374[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/.tmp/info/9b47894aaaef4d06a14b2a585b7402d3 2023-04-27 16:55:34,097 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,44953,1682614416269] 2023-04-27 16:55:34,097 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,44953,1682614416269; numProcessing=1 2023-04-27 16:55:34,098 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:34,097 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:34,098 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2a93512c rejected from java.util.concurrent.ThreadPoolExecutor@46a4d374[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,097 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,44953,1682614416269 2023-04-27 16:55:34,098 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b28a219 rejected from java.util.concurrent.ThreadPoolExecutor@5f173da4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/.tmp/info/9b47894aaaef4d06a14b2a585b7402d3 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/info/9b47894aaaef4d06a14b2a585b7402d3 2023-04-27 16:55:34,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/info/c785e48dc81a447fabc93cf02b5272bb 2023-04-27 16:55:34,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/info/9b47894aaaef4d06a14b2a585b7402d3, entries=2, sequenceid=6, filesize=4.8 K 2023-04-27 16:55:34,111 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c785e48dc81a447fabc93cf02b5272bb 2023-04-27 16:55:34,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a181d38dc7ff4539b60029e08bf407bf in 78ms, sequenceid=6, compaction requested=false 2023-04-27 16:55:34,118 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,44953,1682614416269 already deleted, retry=false 2023-04-27 16:55:34,118 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,44953,1682614416269 expired; onlineServers=2 2023-04-27 16:55:34,118 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,41143,1682614416328] 2023-04-27 16:55:34,118 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,41143,1682614416328; numProcessing=2 2023-04-27 16:55:34,131 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,41143,1682614416328 already deleted, retry=false 2023-04-27 16:55:34,131 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,41143,1682614416328 expired; onlineServers=1 2023-04-27 16:55:34,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/namespace/a181d38dc7ff4539b60029e08bf407bf/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-04-27 16:55:34,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:34,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:55:34,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for a181d38dc7ff4539b60029e08bf407bf: 2023-04-27 16:55:34,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1682614418726.a181d38dc7ff4539b60029e08bf407bf. 2023-04-27 16:55:34,161 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/rep_barrier/1c4fe8221c014c6381e26f76be96c29b 2023-04-27 16:55:34,164 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c4fe8221c014c6381e26f76be96c29b 2023-04-27 16:55:34,191 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/table/90d424791bf24e9d9455630a34453d6c 2023-04-27 16:55:34,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90d424791bf24e9d9455630a34453d6c 2023-04-27 16:55:34,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/info/c785e48dc81a447fabc93cf02b5272bb as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/info/c785e48dc81a447fabc93cf02b5272bb 2023-04-27 16:55:34,201 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c785e48dc81a447fabc93cf02b5272bb 2023-04-27 16:55:34,201 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/info/c785e48dc81a447fabc93cf02b5272bb, entries=59, sequenceid=210, filesize=11.1 K 2023-04-27 16:55:34,201 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/rep_barrier/1c4fe8221c014c6381e26f76be96c29b as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/rep_barrier/1c4fe8221c014c6381e26f76be96c29b 2023-04-27 16:55:34,207 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c4fe8221c014c6381e26f76be96c29b 2023-04-27 16:55:34,208 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/rep_barrier/1c4fe8221c014c6381e26f76be96c29b, entries=24, sequenceid=210, filesize=7.5 K 2023-04-27 16:55:34,208 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,208 INFO [RS:1;jenkins-hbase12:41143] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,41143,1682614416328; zookeeper connection closed. 2023-04-27 16:55:34,208 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@52b04d7c rejected from java.util.concurrent.ThreadPoolExecutor@46a4d374[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,208 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:41143-0x100f3748f680002, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,208 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@53239de7 rejected from java.util.concurrent.ThreadPoolExecutor@46a4d374[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,208 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@54dc405f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@54dc405f 2023-04-27 16:55:34,208 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/.tmp/table/90d424791bf24e9d9455630a34453d6c as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/table/90d424791bf24e9d9455630a34453d6c 2023-04-27 16:55:34,211 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90d424791bf24e9d9455630a34453d6c 2023-04-27 16:55:34,212 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/table/90d424791bf24e9d9455630a34453d6c, entries=36, sequenceid=210, filesize=8.0 K 2023-04-27 16:55:34,212 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 172ms, sequenceid=210, compaction requested=false 2023-04-27 16:55:34,217 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-04-27 16:55:34,217 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:34,217 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-27 16:55:34,218 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-27 16:55:34,218 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-27 16:55:34,218 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-04-27 16:55:34,241 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,46011,1682614416374; all regions closed. 2023-04-27 16:55:34,244 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374/jenkins-hbase12.apache.org%2C46011%2C1682614416374.meta.1682614418282.meta not finished, retry = 0 2023-04-27 16:55:34,308 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,308 INFO [RS:0;jenkins-hbase12:44953] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,44953,1682614416269; zookeeper connection closed. 2023-04-27 16:55:34,308 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@35631442 rejected from java.util.concurrent.ThreadPoolExecutor@5f173da4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,308 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2ed738a6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2ed738a6 2023-04-27 16:55:34,308 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:44953-0x100f3748f680001, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,308 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d59b25c rejected from java.util.concurrent.ThreadPoolExecutor@5f173da4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,347 DEBUG [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs 2023-04-27 16:55:34,347 INFO [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C46011%2C1682614416374.meta:.meta(num 1682614418282) 2023-04-27 16:55:34,357 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/WALs/jenkins-hbase12.apache.org,46011,1682614416374/jenkins-hbase12.apache.org%2C46011%2C1682614416374.1682614418038 not finished, retry = 0 2023-04-27 16:55:34,461 DEBUG [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/oldWALs 2023-04-27 16:55:34,461 INFO [RS:2;jenkins-hbase12:46011] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C46011%2C1682614416374:(num 1682614418038) 2023-04-27 16:55:34,461 DEBUG [RS:2;jenkins-hbase12:46011] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,461 INFO [RS:2;jenkins-hbase12:46011] regionserver.LeaseManager(133): Closed leases 2023-04-27 16:55:34,462 INFO [RS:2;jenkins-hbase12:46011] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-27 16:55:34,463 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-27 16:55:34,463 INFO [RS:2;jenkins-hbase12:46011] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:46011 2023-04-27 16:55:34,541 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,46011,1682614416374 2023-04-27 16:55:34,541 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-27 16:55:34,542 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,46011,1682614416374] 2023-04-27 16:55:34,542 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,46011,1682614416374; numProcessing=3 2023-04-27 16:55:34,562 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,46011,1682614416374 already deleted, retry=false 2023-04-27 16:55:34,562 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,46011,1682614416374 expired; onlineServers=0 2023-04-27 16:55:34,562 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,37667,1682614415067' ***** 2023-04-27 16:55:34,562 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-04-27 16:55:34,563 DEBUG [M:0;jenkins-hbase12:37667] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73fc4cf6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-04-27 16:55:34,563 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,37667,1682614415067 2023-04-27 16:55:34,563 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,37667,1682614415067; all regions closed. 2023-04-27 16:55:34,563 DEBUG [M:0;jenkins-hbase12:37667] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-27 16:55:34,563 DEBUG [M:0;jenkins-hbase12:37667] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-04-27 16:55:34,564 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-04-27 16:55:34,564 DEBUG [M:0;jenkins-hbase12:37667] cleaner.HFileCleaner(317): Stopping file delete threads 2023-04-27 16:55:34,564 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1682614417595] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1682614417595,5,FailOnTimeoutGroup] 2023-04-27 16:55:34,564 INFO [M:0;jenkins-hbase12:37667] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-04-27 16:55:34,564 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1682614417596] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1682614417596,5,FailOnTimeoutGroup] 2023-04-27 16:55:34,564 INFO [M:0;jenkins-hbase12:37667] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-04-27 16:55:34,564 INFO [M:0;jenkins-hbase12:37667] hbase.ChoreService(369): Chore service for: master/jenkins-hbase12:0 had [] on shutdown 2023-04-27 16:55:34,564 DEBUG [M:0;jenkins-hbase12:37667] master.HMaster(1512): Stopping service threads 2023-04-27 16:55:34,564 INFO [M:0;jenkins-hbase12:37667] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-04-27 16:55:34,564 ERROR [M:0;jenkins-hbase12:37667] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-04-27 16:55:34,565 INFO [M:0;jenkins-hbase12:37667] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-27 16:55:34,565 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-04-27 16:55:34,573 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-04-27 16:55:34,573 DEBUG [M:0;jenkins-hbase12:37667] zookeeper.ZKUtil(398): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-04-27 16:55:34,573 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-27 16:55:34,573 WARN [M:0;jenkins-hbase12:37667] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-04-27 16:55:34,573 INFO [M:0;jenkins-hbase12:37667] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-27 16:55:34,573 INFO [M:0;jenkins-hbase12:37667] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-04-27 16:55:34,573 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-27 16:55:34,573 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:55:34,573 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:55:34,573 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-27 16:55:34,573 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:55:34,573 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.81 KB heapSize=719.71 KB 2023-04-27 16:55:34,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-27 16:55:34,601 INFO [M:0;jenkins-hbase12:37667] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.81 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/df6cdc93f51448498f29328c944853f8 2023-04-27 16:55:34,604 INFO [M:0;jenkins-hbase12:37667] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df6cdc93f51448498f29328c944853f8 2023-04-27 16:55:34,605 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/df6cdc93f51448498f29328c944853f8 as hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/df6cdc93f51448498f29328c944853f8 2023-04-27 16:55:34,608 INFO [M:0;jenkins-hbase12:37667] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df6cdc93f51448498f29328c944853f8 2023-04-27 16:55:34,609 INFO [M:0;jenkins-hbase12:37667] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43755/user/jenkins/test-data/f464620a-82e4-5c2d-6b45-56ffbf13fc86/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/df6cdc93f51448498f29328c944853f8, entries=156, sequenceid=1282, filesize=34.9 K 2023-04-27 16:55:34,610 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegion(2948): Finished flush of dataSize ~608.81 KB/623418, heapSize ~719.70 KB/736968, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=1282, compaction requested=false 2023-04-27 16:55:34,612 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-27 16:55:34,612 DEBUG [M:0;jenkins-hbase12:37667] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-27 16:55:34,621 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-27 16:55:34,621 INFO [M:0;jenkins-hbase12:37667] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-04-27 16:55:34,622 INFO [M:0;jenkins-hbase12:37667] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:37667 2023-04-27 16:55:34,634 DEBUG [M:0;jenkins-hbase12:37667] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase12.apache.org,37667,1682614415067 already deleted, retry=false 2023-04-27 16:55:34,652 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,652 INFO [RS:2;jenkins-hbase12:46011] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,46011,1682614416374; zookeeper connection closed. 2023-04-27 16:55:34,652 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@37ee9b48 rejected from java.util.concurrent.ThreadPoolExecutor@33d1196d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 314] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,652 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@11aec341] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@11aec341 2023-04-27 16:55:34,652 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): regionserver:46011-0x100f3748f680003, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,652 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@498bc2e rejected from java.util.concurrent.ThreadPoolExecutor@33d1196d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 314] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,652 INFO [Listener at localhost.localdomain/45365] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-04-27 16:55:34,752 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,752 INFO [M:0;jenkins-hbase12:37667] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,37667,1682614415067; zookeeper connection closed. 2023-04-27 16:55:34,752 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1902c398 rejected from java.util.concurrent.ThreadPoolExecutor@67e4675f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,753 DEBUG [Listener at localhost.localdomain/45365-EventThread] zookeeper.ZKWatcher(600): master:37667-0x100f3748f680000, quorum=127.0.0.1:58499, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-27 16:55:34,753 ERROR [Listener at localhost.localdomain/45365-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@37eb5720 rejected from java.util.concurrent.ThreadPoolExecutor@67e4675f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-27 16:55:34,754 WARN [Listener at localhost.localdomain/45365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-27 16:55:34,759 INFO [Listener at localhost.localdomain/45365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-27 16:55:34,864 WARN [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 5799fd8f-0a4b-4c91-a1ff-1fe687f21837) service to localhost.localdomain/127.0.0.1:43755 2023-04-27 16:55:34,867 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data5/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:34,868 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data6/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:34,870 WARN [Listener at localhost.localdomain/45365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-27 16:55:34,874 INFO [Listener at localhost.localdomain/45365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-27 16:55:34,982 WARN [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-04-27 16:55:34,983 WARN [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid c3e9854d-f6b7-4cf1-bd29-59caf4745df9) service to localhost.localdomain/127.0.0.1:43755 2023-04-27 16:55:34,985 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data3/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:34,986 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data4/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:34,987 WARN [Listener at localhost.localdomain/45365] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-27 16:55:34,989 INFO [Listener at localhost.localdomain/45365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-27 16:55:35,096 WARN [BP-2099223383-136.243.104.168-1682614409898 heartbeating to localhost.localdomain/127.0.0.1:43755] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2099223383-136.243.104.168-1682614409898 (Datanode Uuid 6c9ebd47-0a00-4e93-9558-ec52eb9cbf04) service to localhost.localdomain/127.0.0.1:43755 2023-04-27 16:55:35,099 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data1/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:35,100 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/cfda240c-7ecf-a216-a98b-fc0a36936b1d/cluster_72fde25e-0d37-f0e9-17c1-10021b16da09/dfs/data/data2/current/BP-2099223383-136.243.104.168-1682614409898] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-27 16:55:35,119 INFO [Listener at localhost.localdomain/45365] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-04-27 16:55:35,229 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-27 16:55:35,229 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-27 16:55:35,302 INFO [Listener at localhost.localdomain/45365] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-04-27 16:55:35,536 INFO [Listener at localhost.localdomain/45365] hbase.HBaseTestingUtility(1293): Minicluster is down